I need to be able to generate a 3D perspective from a bunch of 2D images of a pipe.
Basically... We have written software that interprets combined data from laser and sonar units to give us an image slice from a section of pipe. These units travel through the pipe and scan the inside of the pipe every 100mm.
All of this is working great. My client now wants to take all these 2D image slices and generate a 3D view so they can "travel" through the pipe looking at defects etc.. that are picked up by the scans. We can see the defects in the 2D images but there can be hundreds of images in a single inspection - hence the requirement to be able to look through the pipe.
I am doing this in VS2010 on the .NET 4 platform in C#.
I am honestly clueless as to where to start here. I am not a graphics developer so this is all new territory to me. I see it as a great challenge but need some help kicking off - and a bit of direction.
Any help appreciated :)
Mike
Well, every 10cm isn't very detailed.. However, you need to scan the pixels of the pipe, creating a list of closed polygons, then just use a trianglestrip to connect one set to the next, all the way down the pipe.
Try to start with very basic 2d instead of full blown 3d rendering - may be good enough. Pipe when you look at it from inside can be represented as several trapeze. Assuming your images are small cylinder portions of a pipe - map each stripe to trapezoids (4 would be good start - easy to position) and draw than in circular pattern. You may draw several stripes this way at the same time. To move back/forward - just reassign images to trapezoids.
If you need full 3d - consider if WPF would work, if not - XNA or some OpenGL library will give you full 3d.
You don't specify the context, 100mm sample intervals may be sparse (a 1m pipe) or detailed (10km pipe). Nor do you specify how many sample points there are (number of cross sections and size of cross section image).
A simple way to show the data is to use voxels where each pixel on a cross section is treated as a cube and adjacent samples form adjacent cubes (think Minecraft). The result will look blocky but as it's an engineering / scientific application this is probably preferable. Interpolating the model to produce a smooth surface may hide defects or make areas appear to be defective. Also, rendering a cross section through a voxel is a bit easier than a polygon surface.
Related
Here's the deal - I'm working on an algorithm/library that is able to generate a navigation mesh in virtually any environment where I can get coordinates for the controlled agent and/or for other agents within the same static environment. The only input I have, is a collection of points where an agent has been to.
(See the image here to hopefully understand what I mean)
I already got to the point where I can create navmeshes manually and navigate on them well enough. However, in larger environments, having only coordinates of, say, the controlled agent, it's really tedious and time-consuming to manually do it.
The uses for such algorithm/library for me are obvious, but I have put a lot of thought into it already, so I'll list a couple of things I'd like to accomplish:
Robotics (scans environment, only gets distance from self to a point, hence getting coordinates - no need for complicated image/video processing)
AI that is able to navigate an unknown and unseen maze (any shape or size) by exploring it
Recording walked areas and creating AI for games that don't know certain places unless they've been there
Now you hopefully see what kind of solutions I'm looking for.
I have tried a couple of things, but couldn't figure them out. One of the most successful things I've tried is giving a range to each individual point (creating a circle), and then looking for places with overlapping circles - you can most likely move on those areas. The problems with this approach started with triangulation of the areas. The resulting mesh may be a little inaccurate, but it must be able to connect to existing ("discovered") parts of the mesh seamlessly (not everything has to be interconnected somehow, as agents can disappear and reappear, but within reasonable proximity, connect the mesh).
Some more information: I'm working in C#, though solutions in java, C++/C, objective C, pseudocode etc are equally acceptable.
P.S. I'm not interested at all in answers like "just use this library" or "use this other language/environment" etc... I want an algorithm. Thank you in advance.
I can help with 2D path finding. You need to find the red outline. Then you can use a voronoi diagram with the red outline (not the agents points). Remove all edges outside the red outline and the remaining edges can be used to navigate the shape by someone/something. Read about it:http://www.cs.columbia.edu/~pblaer/projects/path_planner/.
These days I am trying to develop two algorithms in c#. Self Organizing Map, Particle Swarm Optimization and Glowworm Swarm optimization. I know how the algorithms work but there is an issue which I am not sure where to start from.
Agents in the search space which try to find the best solution have some coordinates(x and y). I don't know how am I should represent the position of agents visually in a form in each iteration. One option may be using charts in c# and represent point so that in each iteration I am going to change the position of the agent(point) in the chart. Another way may be using drawing classes in c# and drawing circles or points in a panel based on the x and y coordinates. Which classes of .net should I use to represents points in a search space visually(in a 2D space).
I hope you understood me and thank you for reading this post.
If your design variables are N-dimensional, N>3, it is not an easy job to visualize the entire domain of interest. You can either project the N-Dim to 2D or 3D to get a "section" of the fields.
I have never written any silverlight apps but I am looking to write a 3d viewer for earthquakes and have it run from my web site.
I would like to create a simple viewer so the user can change the "camera" ie their perspective. The view could contain up to 10,000 objects in the 3d space.
I want the ability to quickly view this - I have seen this on a Power Basic application and want to do this for the web.
I have a current web site at http://canterburyquakelive.co.nz for earthquakes in Canterbury New Zeaalnd and I want to learn the basics so that it can be more interactive.
I want to say for example (to start) place 2 objects in a "space" that I can define and move the camera in real time.
The system must support up to 10,000 objects in the end of the day.
Each object can be a simple circle - no need for special pixel shaders
I am unsure of the exact functionallity of the system at the moment so if I can find a tutorial that allows me to place someone (a circle) into a 3d world (space) and change the camera that would be good.
Any ideas appreciated - there seems to be so much about 3d and silverlight that I may be getting lost in the "gloss" of new features where I need some basics and I can learn and adapt over time.
** Added comment + image **
Basically I am waiting to create a page that look like this using Silverlight. But I am open to any technology.
I've never done 3D in silverlight so I can't exactly answer your question as asked but in general to display geographic markers in a 'real' 3D terrain is quite involved. At a minimum you're probably looking at:
Obtaining binary height data files (last time I looked, NASA gives this away)
Reading and interpreting said files to get 'bitmap' height data
Choosing and dealing with projections (e.g. UTM)
Deciding how to tesselate your bitmap height data
If you want it textured you'll need to also obtain satellite data for that, again converting or processing it to account for projection.
You could ignore the terrain height, but that may not simplify things depending on how 'bumpy' your terrain is.
For a pre-defined small enough area, you could perhaps pre-author a 3d model of the terrain in some 3D package but displaying your markers will still require a projection from long/lat into your 3D space, and you'll still need to know terrain height (unless you do mesh collision with the static model).
Rendering the markers is pretty straightforward by comparison, choose from:
Use a 3D model e.g. a 'pin head' (simple but not always visible)
Render a regular n-gon with 'viewer facing' polygons (resolution independent but maybe ugly)
Render a quad with a circle texture on it (low poly but what size texture to choose?)
There are probably libraries that do some or all of this for you, so if you are set on rolling your own then some of the things I've mentioned could form the basis for your search.
However, given what you've described of your site and situation I suspect you'd be better off avoiding all that work by using a pre-existing solution. E.g. the Google Earth API.
You could consider 3D web plugins that -granted- take you away from Silverlight but that might speed up your development process. I'm thinking in particular of e.g. the Blender 3D web plugin. I can understand the need to write your own viewer, but think twice before you re-invent the wheel. Good luck!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I have a shopping mall directory image (for example like this: http://www.westfield.com/annapolis/map/ ) and want to make an application like this http://www.youtube.com/watch?v=LNj8D8JKv-4&feature=related where I can draw the path between two locations without needing make many images for the paths.
So, What techniques (Library, programming techniques, softwares .. etc) do you suggest to do this using .NET (Windows Forms/ WPF) application?
EDIT for BOUNTY
I am looking for some start like. I am on 3rd Floor. I have a image of third floor. There is a point of entrance from 3rd floor in map. There are 29 seats in the floor. I want to show with line path, where is somebody's seat. I want to do it through Web App. C# MVC 4.5 Where should I start from ? Any sample code will be very helpful.
I solved similar problem some time ago. In short, I was making a game like Heroes of Might and Magic and since I didn't want to draw background myself I decided to use a static image which I would simply manipulate along with player moves.
So we found an open-source game with a map editor that we used for creating of a map background. Then we created a mapper that would load an image created from the map editor and user would mark places on a grid. The following image is a screenshot from our map mapper.
There was the image, and drawed grid with adjustable size for cells so that we could map objects very precisely. The yellow boxes with an item inside are mapped objects (some of them actually do not give a sense, it is just a proof of concept). When we were content of the result, we would safe position and information about all obejcts to a xml file so that I can be used from our game application.
In our game we have defined a class called TileMap that was aware of all objects (stored as instances of a class Tile) on the map and all players move requests were permitted or forbidden by it.
Finally, this is how I would solve your problem. I have only positive experience with this solution. Create an application that would allow to you to specify where there is a path, where there is a store and how it is called, and so forth. Serialize values to xml or another format you are confident with. Then, create an application that works with these obejcts, define a tile map and you are nearly finished. Now you just need to implement path finding and drawing. Path finding is easy, there are plenty different algorithms with different efficiency and speed. Once you know what tiles you have to cross in order to get into a destination, simply draw stars, arrows or whatever you like above the tiles.
Cons
you need to create two applications
Pros
no need to generate more images, you are drawing on a transparent layer and you can safe already generated paths in memory or file
quite easy implementation, just a lot of code
feel free to choose a technology - we used WinRT, but WPF and even WinForms are also suitable
if you like GUI applications, you will have a plenty of fun while doing this :)
Feel free to ask any design and even implementation details questions .
Prepare the list of all places that you want to be searchable. Give each one an ID. The stairs and elevators should have IDs too.
Create the map floor masks. The easy way is the "coloring book" technique. Create semi-transparent layer in Photoshop and overlay it above the map. Then, draw the walls with some set "color 1". Then, draw each distinct place with a distinct color. Just convert the place ID to Hex and use that hex value as the color. Draw the mask for each floor (in a separate image).
Create the map loader. It should load the mask images and extract the object position and passing ability information. You have to find the position (x, y, floor) of each place by scanning the maps and looking at the pixel color hex value. You have to locate the stairs and elevators, their positions and the floors that they connect.
Implement the pathfinding algorithm like A*. It's quite easy and looks like viscous water flowing to the destination's low point. For each position (x, y, floor) one can move in any of the four directions where there aren't walls. And if there is an elevator at that point, one can move to some other floor. The algorithm can quickly find the best route between any two (x, y, floor) points.
When user gives you the name of the place where he wants to go, you need to find out the user's position coordinates (you may try to use GPS) and the destination position (use the object location table, you prepared when you loaded the map on step 3). Give these two (x, y, floor) points to the pathfinding algorithm to get back the route - the sequence of the (x, y, floor) points leading to the destination.
Analyze the route. You need to scan the route and find out what floors does it pass through. Split the route in chunks where each chunk belongs to a single floor. Now you have a list of floors and the route points for each of the floors.
Visualize the floors and their route parts by drawing the route points over the floor maps. To increase the root point spacing, you may just draw every 10th point or so. With HTML5, the sequence of route points can be drawn as an SVG or canvas overlay on top of the floor map image background.
The best way to set up something like this is to pick up a book on game programming, as it will give you a lot of information on setting up paths around "solid objects". The actual UI technology should not matter as much, so choose WPF or Windows Forms, HTML5 etc. Of the choices you have, I would probably aim for WPF or Silverlight, as it gives you much more flexibility on creating the UI. But I would not be adverse to HTML5 either.
You can definately do the whole thing in WPF, you're looking at drawing simple paths so you need to chunk your project into multiple sub issues:
1) How to draw the UI area (i assume you already know that)
2) How to draw the map, it heavily depends on the map data you have but it could be as simple as a single image, please add more detail for your source
3) How to figure out the path, for this you will need to use some form of pathfinding algorithm (one of the simplest one is A*, but there is a myriad of algorithms for different needs)
4) How to draw the Path, this depends on what you're looking for once again
I know it isn't much of an answer yet but depending on your needs (please add a comment bellow) i'll edit it to help you the best i can.
It would be pretty significant undertaking if you are trying to do this whole thing by yourself. All you have is image of shopping mall. From there you need to convert it in to vector data which in itself a fairly significant project. Then you will need to design navigation algorithms and then you need to setup UX for the whole thing. IMO, all these would require a lot of resources and research if you want to do this nicely and accurately.
Fortunately here is a good news: Google has been trying to do same thing for quite sometime and throwing lot more resources than that are likely at your disposal. Their effort is called "indoor mapping" which you can pretty much leverage out-of-box for your scenario. I'm going to give out here pointers to start you off as you have asked.
First visit Google blog to get familiarize with their indoor maps initiative. Then try out adding your floor plan in Google maps here. You are basically uploading image and aligning with the building in Google maps at this point. Here's another tutorial. Note that this does not make your floor plan navigable and may not show yet users location on it because to do that you need data from wifi/cell towers to triangulate users location on floor plan. We'll go over that next. If you have tons of these floor plans, I'd suggest taking help of mechanical turk or such service to have other humans do it for you cheaply. Google maps allows you to keep floor plan privates by using overlays but likely you don't want to do that so users can access it from anywhere.
Next, you want to make your user locatable on your floor plan. This involves getting data such as wifi/cell tower signals at different points on your floor plan. Google has app for this. And here's little demo. You can also use SketchUp to add vector data and polygons.
Next, you want to embed Google maps in your app so it becomes integral part of your app instead of users having to go through Google Maps website. To do this look at Maps SDK (here's link for iOS, and snippet for indoor maps).
Good luck!
You may easier this job by looking into Google map indoor solution. http://maps.google.com/help/maps/indoormaps/
No programming except a web page is needed.
I want to count number of people crossing a line from either side. I have a camera that is placed on ceiling and shooting for the floor where the line is (So camera sees just top of people heads; and so it is more of object detection than people detection).
Is there any sample solution for this problem or similar problems like this? So I can learn from them?
Edit 1: More than one person is crossing the line at any moment.
If nothing else but humans are subject to cross the line then you need not to detect people you only have to detect motion.
There are several approaches for motoin detection.
Probably the simplest one fits your goals. You simply calculate difference between successive frames of video stream and this way determine "motion mask" and thus detect line crossing event
As an improvement of this "algorithm" you may consider "running average" method.
To determine a direction of motion you can use "motion templates".
In order to increase accuracy of your detector you may try any background subtraction technique (which in turn is not a simple solution). For example, if there is some moving background which should be filtered out (e.g. using statistical learning)
All algorithms mentioned are included in OpenCV library.
UPD:
how to compute motion mask
Useful functions for determining motion direction cvCalcMotionGradient, cvSegmentMotion, cvUpdateMotionHistory (search docs). OpenCV library contains example code for motion analysis, see motempl.c
advanced background subtraction from "Learning OpenCV" book
I'm not an expert in video-based cv, but if you can reduce the problem into a finite set of images (for instance, entering frame, standing on line, exiting frame), then you can use one of many shape recognition algorithms. I know of Shape Context which is good, but I doubt if it subtle enough for this application (it won't tell the difference between a head and most other round objects).
Basically, try to extract key images from the video, and then test them with shape recognition algorithms.
P.S. Finding the key images might be possible with good motion detection methods.