I am trying to develop an AR application in unity using the new AR Foundation.
This application would need to use two features:
It needs to use a large amount of tracking images
It needs to properly identify the tracked image (marker) (Only one image will be visible at the same moment)
What I need is dynamically generate the fiducial markers preferably with the tracking part same for all and only with specific part carrying id of the marker. Preferably the AR code would be similar to the ARToolkit One from this image:
Do these markers work well with ARfoundation (abstraction over ARCore and ARKit)?
Lets say I ll add 100 of these generated codes into the XRImageIs it possible that AR Foundation image targets get "confused" and mixup tracked images? Could in theory i use QR codes as Markers and simply code ID information into the QR code?
In a project I searched for a good way to implement a lot of different markers to identify a vast amount of different real world objects. At first I tried QRCodes and added them to the Image Database in ARFoundation.
It worked but sometimes markers got mixed up and this already happened by using only 4 QRCodes containing words ("left", "right", "up", "down"). The problem was that ARFoundation relies on ARCore, ARKit, etc. (Depending on the platform you want to build.)
Excerpt from the ARCore guide:
Avoid images with that contain a large number of geometric features, or very few features (e.g. barcodes, QR codes, logos and other line art) as this will result in poor detection and tracking performance.
The next thing I tried was to combine OpenCV with ARFoundation and use ArUco Marker for detection. The detection works much better and faster than the Image Recognition. This was done by accessing the Camera Image and using OpenCVs marker detection. In ARFoundation you can access the camera image by using public bool TryAcquireLatestCpuImage(out XRCpuImage cpuImage).
The Problem of this method:
This is a resource-intensive process that impacts performance...
On an iPad Pro 13" 2020, the performance in my application dropped from constant 60 FPS to around 25 FPS. For me, this was a too serious performance drop.
A solution could be to create a collection of images with large variations and perfect score, but I am unsure how images with all these aspects in mind could be generated. (Probably also limited to 1000 images per reference database, see ARCore guide)
If you want to check if these markers works well in ARCore , goto this link and download the arcoreimg tool.
The tool will give you a score that will let you know if this image is trackable or not. Though site recommends the score to be 75 , i have tested this for score of as low as 15. Here is quick demo if you are interested to see. The router image in the demo has a score of 15.
Related
Currently, I have a relatively simple 2D game which should not be too taxing on a cpu. It runs fine on my computer however when I build it to my iPhone or iPad the game starts to be quite jittery and has a poor frame rate. Does anyone have any suggestions on how I could improve the performance (I already use object pooling et cetera I think it has something to do with my draw calls / graphics)
EDIT: turns out the renderer is using most of my cpu
If the issue is as you say too many draw calls then one simply way to reduce those is to pack your sprites using the Sprite Packer. This places all your sprites tightly which reduces the amount of draw calls, if they don't all fit nicely or there's too many of them then they will be split into subpages. If you'd like to see how to use the Unity texture packer in simple steps I would direct you to this blog post that talks about it in more depth. Here's a simple step by step guide that I based off of the blog post linked.
Step 1 Select all those sprites that you want to pack in together
Step 2 Give a Packing Tag as shown in the picture below:
Step 3 Open Sprite Packer window and perform packing
Step 4 Click the Pack button as seen above
There are also third party tools like Texture Packer that can do this, and they have more complex algorithms which gives them the edge over Unity's built in verison.
It's also possible there are issues with your games performance not relating directly to draw calls, I would recommend taking a look at the official Unity guide for mobile optimisation.
I need to be able to generate a 3D perspective from a bunch of 2D images of a pipe.
Basically... We have written software that interprets combined data from laser and sonar units to give us an image slice from a section of pipe. These units travel through the pipe and scan the inside of the pipe every 100mm.
All of this is working great. My client now wants to take all these 2D image slices and generate a 3D view so they can "travel" through the pipe looking at defects etc.. that are picked up by the scans. We can see the defects in the 2D images but there can be hundreds of images in a single inspection - hence the requirement to be able to look through the pipe.
I am doing this in VS2010 on the .NET 4 platform in C#.
I am honestly clueless as to where to start here. I am not a graphics developer so this is all new territory to me. I see it as a great challenge but need some help kicking off - and a bit of direction.
Any help appreciated :)
Mike
Well, every 10cm isn't very detailed.. However, you need to scan the pixels of the pipe, creating a list of closed polygons, then just use a trianglestrip to connect one set to the next, all the way down the pipe.
Try to start with very basic 2d instead of full blown 3d rendering - may be good enough. Pipe when you look at it from inside can be represented as several trapeze. Assuming your images are small cylinder portions of a pipe - map each stripe to trapezoids (4 would be good start - easy to position) and draw than in circular pattern. You may draw several stripes this way at the same time. To move back/forward - just reassign images to trapezoids.
If you need full 3d - consider if WPF would work, if not - XNA or some OpenGL library will give you full 3d.
You don't specify the context, 100mm sample intervals may be sparse (a 1m pipe) or detailed (10km pipe). Nor do you specify how many sample points there are (number of cross sections and size of cross section image).
A simple way to show the data is to use voxels where each pixel on a cross section is treated as a cube and adjacent samples form adjacent cubes (think Minecraft). The result will look blocky but as it's an engineering / scientific application this is probably preferable. Interpolating the model to produce a smooth surface may hide defects or make areas appear to be defective. Also, rendering a cross section through a voxel is a bit easier than a polygon surface.
I have never written any silverlight apps but I am looking to write a 3d viewer for earthquakes and have it run from my web site.
I would like to create a simple viewer so the user can change the "camera" ie their perspective. The view could contain up to 10,000 objects in the 3d space.
I want the ability to quickly view this - I have seen this on a Power Basic application and want to do this for the web.
I have a current web site at http://canterburyquakelive.co.nz for earthquakes in Canterbury New Zeaalnd and I want to learn the basics so that it can be more interactive.
I want to say for example (to start) place 2 objects in a "space" that I can define and move the camera in real time.
The system must support up to 10,000 objects in the end of the day.
Each object can be a simple circle - no need for special pixel shaders
I am unsure of the exact functionallity of the system at the moment so if I can find a tutorial that allows me to place someone (a circle) into a 3d world (space) and change the camera that would be good.
Any ideas appreciated - there seems to be so much about 3d and silverlight that I may be getting lost in the "gloss" of new features where I need some basics and I can learn and adapt over time.
** Added comment + image **
Basically I am waiting to create a page that look like this using Silverlight. But I am open to any technology.
I've never done 3D in silverlight so I can't exactly answer your question as asked but in general to display geographic markers in a 'real' 3D terrain is quite involved. At a minimum you're probably looking at:
Obtaining binary height data files (last time I looked, NASA gives this away)
Reading and interpreting said files to get 'bitmap' height data
Choosing and dealing with projections (e.g. UTM)
Deciding how to tesselate your bitmap height data
If you want it textured you'll need to also obtain satellite data for that, again converting or processing it to account for projection.
You could ignore the terrain height, but that may not simplify things depending on how 'bumpy' your terrain is.
For a pre-defined small enough area, you could perhaps pre-author a 3d model of the terrain in some 3D package but displaying your markers will still require a projection from long/lat into your 3D space, and you'll still need to know terrain height (unless you do mesh collision with the static model).
Rendering the markers is pretty straightforward by comparison, choose from:
Use a 3D model e.g. a 'pin head' (simple but not always visible)
Render a regular n-gon with 'viewer facing' polygons (resolution independent but maybe ugly)
Render a quad with a circle texture on it (low poly but what size texture to choose?)
There are probably libraries that do some or all of this for you, so if you are set on rolling your own then some of the things I've mentioned could form the basis for your search.
However, given what you've described of your site and situation I suspect you'd be better off avoiding all that work by using a pre-existing solution. E.g. the Google Earth API.
You could consider 3D web plugins that -granted- take you away from Silverlight but that might speed up your development process. I'm thinking in particular of e.g. the Blender 3D web plugin. I can understand the need to write your own viewer, but think twice before you re-invent the wheel. Good luck!
I am making an object tracking application. I have used Emgucv 2.1.0.0
to load a video file
to a picturebox. I have also taken the video stream from a web camera.
Now, I want
to draw an unfilled square on the video stream using a mouse and then track the object enclosed
by the unfilled square as the video continues to stream.
This is what people have suggested so far:-
(1) .NET Video overlay drawing(DirectX) - but this is for C++ users, the suggester
said that there are .NET wrappers, but I had a hard time finding any.
(2) DxLogo sample
DxLogo – A sample application showing how to superimpose a logo on a data stream.
It uses a capture device for the video source, and outputs the result to a file.
Sadly, this does not use a mouse.
(3) GDI+ and mouse handling - this area I do not have a clue.
And for tracking the object in the square, I would appreciate if someone give me some research paper links to read.
Any help as to using the mouse to draw on a video is greatly appreciated.
Thank you for taking the time to read this.
Many Thanks
It sounds like you want to do image detection and / or tracking.
The EmguCV ( http://www.emgu.com/wiki/index.php/Main_Page ) library provides a good foundation for this sort of thing in .Net.
e.g. http://www.emgu.com/wiki/index.php/Tutorial#Examples
It's a pretty meaty subject with quite a few years and different branches of research associated with it so I'm not sure anyone can give the definitive guide to such things but reading up neural networks and related topics would give you a pretty good grounding in the way EmguCV and related libraries manage it.
It should be noted that systems such as EmguCV are designed to recognise predefined items within a scene (such as a licence plate number) rather than an arbitory feature within a scene.
For arbitory tracking of a given feature, a search for research papers on edge detection and the like (in combination with a library such a EmguCV) is probably a good start.
(You also may want to sneak a peek at an existing application such as http://www.pfhoe.com/ to see if it fits your needs)
Problem Overview
I am working on a game application and need to be able to implement scrollable maps in Silverlight similar to those found in Google Maps. However, I am unsure as to how to implement this effectively. The following paragraphs provide much additional detail. Any ideas or guidance is greatly appreciated!
Problem Detail
I have been working on a new MMOG (massively multi-player online game). The game will implement a coordinate (x,y) based map. Only a very small fraction (less than 0.1%) of the map will be displayed on the screen at any given time. The player should have the ability to click on the map and drag the mouse to scroll and view map areas which are not presently visible. (This is somewhat similar to Google Maps.)
The map background is made up of a series of stitched (repeating) images. These images are woven together to give the basic appearance of the game's "world". A standard set of additional graphics are then superimposed, as appropriate, on each of the coordinate locations . For example, point (0,0) might be a lake, (0,1) might be a city, and (0,2) might be a forest. The respective images for a lake, a city, and a forest would be superimposed on the background.
It is important to mention that the entire map is NOT stored on the local client machine. Rather, as a player scrolls to or opens a specific location, the appropriate map information is retrieved from the remote game server. It is infeasible for us to build the entire game world map ahead of time due to its size and the fact that portions of the map are constantly changing.
I have toyed with the idea of building a bitmap on-the-fly of the new map each time a player moves. However, I think there may be a much better way to add to the map as the player scrolls.
When scrolling, movement of the map should not, if possible, result in a "flickered" refresh of the screen. I believe recreating a bitmap each and every time a player moves even one or two pixels would almost certainly result in flicker.
I am open to 3rd party tools and solutions. However, to the degree possible, I would prefer to use standard Microsoft libraries or open source tools rather than commercial tools.
What are some ideas as to the best way to implement this functionality so that it performs well, is reliable, and transitions to new areas of the map appear seamless to the player?
Thank you in advance for all your help!
Update
Here are a few pieces of additional information that may prove helpful.
Since my initial post, I have been introduced to the concept of a "tile engine". (Many thanks to Michael and Paul for pointing me towards Bing and BruTile.)
My understanding is that a tile engine basically breaks larger images into sections and renders them side by side. As a user scrolls, additional tiles are rendered as others are removed from view. This is very much what I am looking for.
However, there may be a couple of wrinkles that affect my use of a standard tile engine. All of the graphics for the game, including the backgrounds which would be displayed on any tile, will already be downloaded on the client. It is important that the tile engine not retrieve the graphics from a server as this would consume significant unnecessary bandwidth.
Other graphics (e.g. a lake, forest, hill), which represent objects from the gameworld, must be superimposed when the tiles are rendered on the screen. Tile engines such as Bing appear to provide the ability to superimpose custom images. Whatever tile engine is used must not only support this feature but allow exact placement of these superimposed images.
Finally, there is a a requirement to support popup descriptions when the user mouses over one of the superimposed graphics. Unlike the graphics which are already stored on the client, the descriptions contain information which must be downloaded from the game server. BruTile, while excellent in many ways, does not appear to yet support these popup descriptions.
We are making great progress. Thanks for all your help so far!
For an open source solution you could look at BruTile. It too has all the features you describe. It can also be used on the Microsoft Surface and on Windows Phone (for your markeplace version).
Use the Bing Maps control or the MultiScaleImage (Deep Zoom) which it uses.
To seen an example, go here. You can use the Deep Zoom Composer to create maps or topologies using your own photos and images.
Here is the SDK for the control.