I am responsible for a team of developers who will are about to start development of a light weight insurance claims system. The system involves a lot of manual tasks and business workflows and we are looking at using Windows Workflow (.NET 4.0).
An example of the business domain is as follows:
A policy holder calls the contact centre to lodge a claim. This “event” fires two sub tasks which are manually actioned in parallel and may take a lengthy time to complete;
Check customer for fraud – A manual process whereby an operator calls various credit companies to check and assess the potential of a fraudulent customer. From here the sub task can enter a number of sub-statuses (Check in progress, Failed Reference Check, Passed Reference Check, etc)
Send item to repairs centre – A manual process where the item for which the policy holder lodged the claim is sent the repairs centre to be fixed. From here the sub task can enter a number of sub-statuses (Awaiting Repair, In Progress, Repaired, Posted, etc).
The claim can only proceed once the status of each sub task has reached a predefined status (based on the business rules).
On the surface it seems that Workflow is indeed the best technology choice; however I do have a few concerns in using WF 4.0.
Skill set – Looking at the average developer skill set I do not see many developers who understand or know Workflow.
Maintainability – There seems to be little support within the community for WF 4.0 projects and this coupled with the lack of skill set raise concerns around maintainability.
Barrier to entry – I have a feeling that Windows Workflow has a steep learning curve and it’s not always that easy to pick up.
New product – As Workflow has been completely rewritten for .NET 4.0 I see the product as a first generation product and may not have the necessary stability.
Reputation – Previous versions of Workflow were not well received, considered difficult to develop with and resulted in poor business uptake.
So my question is should we use Windows Workflow (WF) 4.0 for this situation or is there an alternative technology (e.g., Simple State Machine, etc) or even a better workflow engine to use?
I have done several WF4 projects so lets see if I can add any useful info to the other answers.
From the description of your business problem it sounds like WF4 is a good match, so no problems there.
Regarding your concerns you are right. Basically WF4 is a new product and is lacking some important features and has some rough edges. There is a learning curve, you do have to do some things differently. The main point is long running and serialization, which is something the average developer is not used to and requires some thought to get right as I hear far too often that people have problems serializing an entities framework data context.
Most of the time using workflow services hosted in IIS/WAS is the best route when doing these long running type of workflows. That makes solving the versioning problem not to hard either, just have the first message return the workflow version and make that a part of each subsequent message. Next put the WCF router in between that routes the message to the correct endpoint based on the version. The basic is never to change an existing workflow, always create a new one.
So what is my advise to you?
Don't take a big gamble on a unknown, and for you unproven, piece of technology. Do a small, non critical, piece of the application using WF4. That way if it works you can expand on it but if it fails you can rip it out and replace it with more traditional .NET code. That way you get real experience with WF4 instead of having to base a decision on second hand information and you learn a new and powerful technology in the process. If possible take a course on WF4 as that will save you a lot of time in getting up to speed (shameless self plug here).
About the Simple State Machine. I have not used it but I was under the impression it was for short running, in memory, state machines. One of the main benefits of WF4 is the long running aspects.
I have come to this dilemma couple of times and I had chosen not to use Work Flow foundation. Some of considerations (similar to yours) were
Involved work flows were lot simpler (a combination of state machine and sequential actions) and doing it in WF seems to overkill for efforts involved.
Learning curve for developers to understand and to use WF effectively was considered high. Status transition table describing valid transitions and actions to be taken are used for additional flexibility and developers were comfortable with it, easily understanding the concept and purpose.
Chances of business process changes were slim and rudimentary changes were easily possible with help of transition table. A change in transition would mean a database script while change in actions would result in new release/patch. However, probability of such occurrence was deemed to be low.
Looking back after 13-14 months, I still think that decision of not using WF was correct. IMO, WF makes sense where there is strong likely hood that work flow can change and/or business rules can change. WF allows to isolate workflow in separate file and so making it configurable by users will be simpler.
We have been using WF 4.0 the last couple of months. I have to say it's challenging to think the Workflow way. However, I can tell you it's worth it. We knew very little when we started. We've bought a beginner and professional book for WF 4.0 that helped. I, myself, watched many videos online and followed PDC 2009 for their breaking news about WF 4.0 and how it's different from the previous somewhat sucky versions.
One major thing that we had to propose a solution for is the way we can deal with In/Our Arguments in a workflow without bounding our custom activities to certain data types and how to pass parameters between activities. I have come up with a good solution for that, and the workflow experience that we have so far is not bad at all. Actually, we have a workflow-intensive application that is getting bigger and bigger and I really cannot imagine myself solving it in a different environment. I love the visual effect that it has: it keeps me away from the details of if/else etc constructs and makes the business rules apparent in a way that doesn't make you forced to dive into lines of code to know what's going on or how to fix some bug.
By the way, the project that we worked on is very similar to what you described and it's a medium-sized project.
You can tell from my words that I like it and I do recommend it although is incorporates some risks as it's a new technology and you have to come up with some innovative ideas.
my 2 cents...
I did three projects in WF 3.5 and I have to say it is not easy. It force you to think in the whole new way especially when persistance is used. Updating the application which contains hundreds of incomplete persisted workflow is challenging. Single breaking change in serialization crashes them all. Introducing multiple versions of the same library to support new and old running workflows is common. It was challenging.
I haven't tryed WF 4.0 yet but based on experience from BizTalk and WF 3.5 I think it will be similar.
Anyway the best approach you can take is to do Proof-of-Concept. Take single WF from your requirments and try to implment it in WF 4.0. You will spend some time with it but you will prove if you are able to do that in WF 4.0 and if there are any visible benefits.
If you decide to use WF 4.0 I insist that you check possibility to run WF as WCF service hosted in Windows AppFabric. AppFabric provides some out of the box functionality for hosting WFs.
I think it does not really make sense today to talk about Workflow in WF4 as a technology choice for this kind of problem. What is really appropriate, as mentioned by Ladislav Mrnka above, is WCF WF Services hosted in AppFabric.
My experience with this is that it pays great dividends and is very enjoyable, but problems arise in the beginning because it is not properly appreciated that for many programmers this is a methodology shift more than a technology shift. On the other hand, generalists and those with a problem-solving mindset saw WCF WF AppFabric as a set of exciting opportunities. So if the mix of people on the project are fairly conservative C# devs attached to their daily set of OO and patterns, it will be hard to introduce. If the team is more innovative, then adoption will be much easier because the potential and new doorways multiply with each discovery.
Two main conceptual problems programmers had in moving to this technology was:
a) Message correlation and messaged exchange patterns
b) Workflows and unit testing
In standard systems in C# for example a workflow is rarely explicit and therefore rarely unit tested. The overall workflow is left for testing by acceptance scenarios or integration. Introduce an explicit WF as a software artifact and suddenly standard devs want to try and unit test it, which is usually not worth doing.
The message correlation aspect of it is a bit of mindset shift for those not familiar with message exchange patterns. Most devs have dealt with in process and remote calls, web service and SOAP, and usually focussed on one or two of those. To abstract above it all and work with a general message based system can be confusing at first.
On the positive side though, the end result is something that saves a lot of time and creates a lot of opportunities. One main thing is that the worfklow, if visually clear, is something that can be worked on by end user, developer and analyst together, eliminating unnecessary steps in the development lifecycle and focusing the parties on one artifact. Further, it discourages islands of functionality in dedicated apps, with dedicated glue layers, by encouraging a suite of business processes in WF per business domain. Further, with AppFabric, the plumbing for persistence, logging, and waking up scheduled activities is all done for you. WF4 performance is outstanding too.
My recommendation would be to find the most innovative or explorative team member do the initial scouting to discover the tricky parts, get the core functions working, and have that initial person be responsible for then compartmentalising the remaining work.
In order to do an insurance claim system of any complexity that involves roles and "sub-tasks" you really need an BPM solution, not just workflow. Workflow Foundation 4.0 is slick but it really doesn't not come close to the functionalities of a BPM product.
BPM solutions, like Metastorm BPM, Global360, and K2.NET, provide human centric workflow, tasks, roles, and system integration that can model and streamline the business processes like your insurance claim system. Use ASP.NET to build the interface that integrates with the BPM workflow engine as their built in designers are usually limited and force you to use their custom built web control which usually are not as full featured as the ASP.NET web controls.
Go with the technology your team knows and feels comfortable with. Workflow Foundation is not a product that you can use straight away - it's rather a set of pieces you can embed in your application in order to build a workflow system. IMHO the workflow logic is the least important piece of technology, first of all you have to concentrate on the GUI because business owners will not see anything but the GUI. But if your system is a success then you have to be prepared for neverending change requests and new requirements so you have to implement your business logic so that it's easy to change and easy to divide into separate processes to suit different user needs (sometimes contradicting). BPM helps in this task because it allows you to have separate, multiple versions of business processes suiting various business needs. You don't need full fledged BPM engine for that but it's useful to code your business logic so that it can be versioned and divided into individual business processes - the worst thing to have is an unmantainable and intertangled blob of code that handles 'everything' and that no one can understand. There are many ideas for that - state machines, DSLs (domain specific languages), scripts etc - you decide what the implementation should be. But you should always think in terms of business processes and organize your logic accordingly so that it reflects these processes.
And be prepared for coexistence of many variants of business logic and data structures - this is the most difficult design task imho.
I'm in a situation where I have to use 4.0 as .NET 4.5 isn't accredited for use in our prod environment yet. I had major pain understanding generally how to get long running workflows going to suit our business need but eventually found an elegant solution. It's not something which just anyone coming later to support can just pick up with ease because there's so much to think about, but I do believe in WF as a tool for managing workflow states.
One big thing I take issue with WF 4.0 though is Maurice's comment:
The basic is never to change an existing workflow, always create a new one
That's great if you just want a new version, but what if you have 50,000 persisted workflows and realise at some point that there's a bug in the workflow? You need to be able to update the xamlx and still be coupled to the existing instances. I've tried ungzipping the various metadata columns in the SQL Server instances table to find something that ties the instance to the workflow definition without any luck.
I did write a synchronisation application for importing data from an old system into our new WF 4.0 driven one. We basically load the data into the system, then run the process which goes about automatically calling into the workflow steps and calling validation methods, essentially mocking user interaction. This only really worked well with us due to the architecture we implemented for access to the workflow service host. It's great as a one off, where after running you can go through and do checks to ensure consistency of the data migration process, but having to use this approach for potentially hundreds of thousands of cases once a system is live isn't really an approach that instills confidence and over burdens the process of integration simple bug fixes.
My recommendation is that you avoid WF 4.0 altogether and just go straight to 4.5 if you're environment supports it. The Dynamic Updates and Side by Side Versioning it provides caters for bug fixing and WF versioning all out of the box. I've still yet to investigate exactly how as 4.5 still isn't accredited for use by our client, but eagerly awaiting this opportunity.
What I'm desperately hoping for is that our client doesn't request changes to policy (and therefore workflow adjustments) and that the current workflows hold up without any bugs. The latter being a vain and empty hope as bugs always pop up.
I really can't understand what was going through the WF dev team's heads to release a system where out of the box you can't fix bugs easily. They should have developed a technique for re-binding an instance to new xamlx.
Related
In our company we develop and sell a VB6 application and we think it's about time to migrate it to .NET.
The main reasons are:
We expect VB6 runtime support to end at some point in time, and we do not want to start the migration just then since it's probably gonna be a lengthy process.
There is just 1 1/2 VB6 developers left. The half one being me.
More and more customers asking for features like cloud and mobile device support.
I know that rewriting an application from scratch is the least recommended way for migrating to .NET. I totally aggree with that! Throwing away over a decade of code feels just wrong and would be such a waste of money spent, that I have a hard time recommending and justifying it towards our management.
But right now I don't see another way to do it.
Let me tell you a little bit about the application:
Like I said it has been developed for over a decade. There have been numerous developers working on it, most of them rather unexperienced at that time. We have one developer left from the initial team. That application has been his first and biggest software project and by now he realizes that many of the architectural decisions made over last 15 years have been horribly wrong, others were right at that time but have not been refactored to meet changes made in other parts of the application and so have become wrong at some point in time. This application seems to be a showcase example of code rot.
We are talking about an application of about 150 KSLOC, all in one single executable. It uses about 15 external DLLs, some of them third party ActiveX controls, some of them are our own .NET assemblies.
Adding new features to the application is still possible and being done, but takes ages compared to our other .NET applications. The reason is that every little change in the codebase requires changes all over the place. The only reason why changes are possible at all is because that one developer simply knows most the dependencies and quirks of the application. As you might have guessed the rate of unexpected side effects and bugs is quite high.
My first thought about migrating that application was to first clean up and refactor, then migrate/convert possibly using tools from Artinsoft/Microsoft/WhoEver and then refactor again to get a nice and clean .NET application.
But I see some problems:
There seems to be no way of refactoring the old application. There is no automated testing whatsoever, not even a formal method for manual testing. Every little change requires manual testing by experienced users who just know where defects might hide.
on the other hand I have established a process and set of tools for testing of our .NET applications which gives us a solid base for making refactorings
Converting that code to .NET without major refactoring feels like: Garbage in, garbage out. Even though I hate calling the old application garbage because somehow it works and has proven itself useful.
Our management has a habit of explicitly demanding quick and dirty solutions, disregarding the effects it has on the productivity and against all recommendations from the development team which has at some point started to deny the existence of quick and dirty solutions in order to be able to do things right. That does not mean that we polish features, but we do include the time to write tests and do refactoring in our estimates. So knowing this, I suspect that once the code is converted to .NET and fixed to the point where the application starts and seems to work, the refactoring-phase will be canceled and the application will be shipped to some customers.
So. What I think is that, despite the fact that rewriting from scratch will take a lot of time and resources, it might still be our only option.
Am I missing an option? Do you see possibilities of not having to rewrite that application?
I suggest that you take a step back and read this paper by Brian Foote & Joseph Yoder (University of Illinois). It provides some architectural insight into the problem you have and options to solve it. It's titled 'Big Ball of Mud' (please don't laugh, it is a serious paper). Here is the abstract:
While much attention has been focused on high-level software
architectural patterns, what is, in effect, the de-facto standard
software architecture is seldom discussed. This paper examines the
most frequently deployed architecture: the BIG BALL OF MUD. A BIG BALL
OF MUD is a casually, even haphazardly, structured system. Its
organization, if one can call it that, is dictated more by expediency
than design. Yet, its enduring popularity cannot merely be indicative
of a general disregard for architecture.
These patterns explore the forces that encourage the emergence of a
BIG BALL OF MUD, and the undeniable effectiveness of this approach to
software architecture. In order to become so popular, it must be doing
something right. If more high-minded architectural approaches are to
compete, we must understand what the forces that lead to a BIG BALL OF
MUD are, and examine alternative ways to resolve them.
A number of additional patterns emerge out of the BIG BALL OF MUD. We
discuss them in turn. Two principal questions underlie these patterns:
Why are so many existing systems architecturally undistinguished, and
what can we do to improve them?
BTW, I think your best option is to use the current application as your Requirements and rewrite everything in VB.NET or C# using a proper design.
There are four main options when you have an application like this:
Do nothing: this is always an option, as everybody knows, if it ain't broke don't fix it. However this might not be an option for several reasons such as needing to comply with some security requirements at the company, or simply because one of the components doesn't work in new platforms.
Rewrite: This would be the dream, right? being able to get rid of all the bad practices and duplicated code and so on? Well, it might be that way, however you have to think all the risks involved in developing a new application from scratch. Do you have all the formal requirements? what about test cases? do your team know every little detail in the code or would you need to go line by line trying to figure out how why that if is there? Also, how many bugs do
Buy something off-the-shelf: Since you are an ISV this won't be an option.
Migrate: Of course you'll be bound by the programming practices you used for the original development but you'll get to a new platform faster, all your business logic will be automatically migrated, you can actually hire developers for the new platform and you can get rid of the legacy elements. From here you can also take advantage of all the tools available to refactor code, continuous integration, unit testing, etc.
Also, with an automatic migration you can actually go further than just WinForms. There are also tools that can take your C# code all the way to the web using a modern architecture.
Of course, I work for Mobilize.Net (previously Artinsoft) and this is my biased perspective.
We've been working on this for around 15 years and have seen dozens of clients who come to us after trying to re-write their application and fail after months or even years of struggling without being able to deliver a working application.
I have a client/server application written in C#/.NET 3.5 that I want to do a bit of performance testing on. I've been looking for a generic framework to help me but not had much luck. I would like something that can manage a set of clients and perform random actions for them based on some settings. I would also then like to record some data relating to this to help me work outsome rough thresholds for my system, e.g. I can support n users performing x actions per second.
I would write code specific to my application to perform tasks such as:
Login/logout a client.
Send messages to the server to perform various actions.
Record acknowledgements and other messages from the server.
Measure statistics specific to the system.
I'm hoping the framework will then be able to take a set of parameters to describe a testing scenario such as:
Number of clients logged in at a given time.
Perform a given number of actions per second for each client.
It would then run the scenario, manage and track all of the users and actions and collate all of the data. (This is the boring bit I'm trying to avoid coding myself...) Ideally it would have some general measurements built in, e.g. time between sending a message and receiving a response, but I could code them myself if not.
I don't want to do any profiling of my code; I can always attach a profiler whilst running these tests later on. Instead I want to make some rough conclusions about my system, i.e. how many users can I throw at it before it breaks. (If there is a better term for this than 'performance testing' please let me know... Stress testing maybe?)
I realise I'm not giving very many specifics about the system here. It strikes me as a fairly general situation - I'm sure there are lots of client/server systems out there that people need to do similar tests on. I've found lots of web based frameworks to do similar things but they seem to be pretty web ingrained and don't lend themselves easily to non-HTTP based systems.
Anyone know of anything that might help? My searching hasn't found anything yet. I should point out that I'm stuck with Visual Studio 2008 Professional for the foressable future so if 2010 can do this it's out of bounds for me. I guess it doesn't have to be a .NET framework provided I can still plugin my .NET code fairly easily.
EDIT To be clear my application isn't a website, it's a Windows Forms client application that connects via a custom protocol to a .NET service. I can write code to perform the relevant client actions, I just need a framework to put it in.
The keyword you are looking for is "Distributed testing".
Smart Bear have a product called TestComplete which supports distributed testing. I don't think it can run multiple instances of your client on a single machine though (maybe it can, but I guess it's not a good idea any way since it would impact the performance results).
They also have an open source project called LoadUI, it is built to integrate with SoapUI, however you might be able to hook it up to your own client-test tool. I have no idea how much effort that would cost.
These are the tools I know of, but there are many more distributed testing tools out there. While most are indeed for web-bases testing, they often are extensible enough to simply kick off a different (GUI-based) testing framework (my favorite is QAliber) which runs the tests on your client app.
To my knowledge, such a framework does not currently exist (and it is likely it won't exist because everyones scenarios are so different). Because "Performance" means different things to different people and different projects, you will usually have to roll your own. Thankfully, with the advent of .NET4 and the TPL this has become easier than ever before.
There are three aspects a Performance test needs to cover:
Define "Load"
Measure "Performance"
Evaluate the Results
As you can imagine, both of these are totally lax definitions. Load could be measured in Requests per Second, Logged In Users etc... Performance could be measured in Response Time, Memory Usage... you get the picture.
So your first step is to define a "unit load", that you can easily scale up. For example you could create a ClientUnitLoad class, that simulates a client and periodically performs actions against your service. The TPL makes it easy to scale your ClientUnitLoad (except you are bound by hardware constraints of course).
The next step is to measure Performance. How you measure heavily depends on the metric you want to collect: Simple Stopwatching is fine 90% of the cases but can only measure time. Building a custom tracing and metrics collection infrastructure is definitely worth it, as you will also need it in production to verify your system performs as expected in the real world.
The last step is to evaluate your Metrics. The largest issue with Performance Tests is that: a) they are slow, and b) unreliable. Theres no way around that. In general, you need large sample sizes (repeated test runs) and some sort of statistical analysis on your metrics to ignore outliers. The processed results should then be compared to a configurable performance baseline.
Depending on the desired level of sophistication, that performance baseline may have simple acceptance criteria (such as: Repsonse Time under 100ms) or advanced, specific criteria (such as: Repsonse Time does not decrease by more than 10% when the amount of logged in users is 10 vs. 1).
I've not tried it myself but this may be suitable for your needs.
https://browsermob.com/performance-testing
Can't really say if there is a testing framework for such a specific purpose, but maybe you could use a simple console application (since the main idea is to test server performance)? You could make multiple calls to the server in different threads and use, for example, Stopwatch class to measure response times.
Or, you could try overloading the service by just increasing the number of calls (likely the service will slow down before threadpool runs out of threads?).
Kindly check at the following link.This may help you.
http://msdn.microsoft.com/en-us/library/aa337591.aspx
I guess you will get the feature even in VS2008 but for ultimate edition.
http://blog.maartenballiauw.be/post/2008/02/code-performance-analysis-in-visual-studio-2008.aspx
Available in Developer and Team-based Visual Studio 2008. I think the feature has been improved in Visual Studio 2010 though.
I should clarify - this doesn't provide support for all the test cases you want to run. You would have to write that logic yourself in a separate project, set that as the startup project, then run the Performance Analysis.
I've done some performance tests a while ago in a client/server application using Microsoft Visual Studio Load Test
When you add unit tests to a load test, you exercise the performance
of non-Web based server components
You have to code your load generators as "Visual Studio Unit Tests", so that later you can add them into the Load Test project.
I have to create a workflow in C# capable of moving an object (persisted as a database record) through an approval workflow where people are required to perform some sort of action or validation.
We initially looked at Windows Workflow Foundation but shied away from it because it seemed so infrastructure-heavy (and besides we don't really like Microsoft products). We looked at ObjectFlow because it's lightweight, but I'm having trouble figuring out how to persist & resume workflow states. It almost seems like it's too lightweight.
Does anyone have a particular favorite framework for doing workflow? I'm open to ideas (even to WWF, if you can explain why it's your favorite).
Now at the end of 2022 I recommend Elsa Workflows library which is free and open source.In this case state machine workflows works. I have used it in multiple projects witch success. It is flexible, has a web workflow designer and acceptable documents
As the question #gsharp linked to says, WF 4 isn't entirely easy to use. However, ObjectFlow has an easy fluent interface that is light and built with solid design principles. Given the apparent lack of decent workflow frameworks, I decided to pitch in and extend ObjectFlow with an IStatefulWorkflow that contains a .Yield() method capable of yielding workflow processing to the calling method so that it's state can be persisted.
The end result of my work will be a new release at codeplex in a couple weeks. Until then, you can follow my progress at github.
Have you looked at Drools.Net?
Drools.NET is a Business Rules Engine (BRE) based on Charles Forgy's Rete algorithm. Developers can now exploit a powerful Rule Engine through a completely managed .NET code base! Drools.NET is based on Jboss Rules, and comes with all the features of that Rules Engine.
I have recently developed a C# Workflow library that leverages the fluent syntax and provides compile time validation between workflow steps. A Workflow's steps can be consolidated in a single location providing maintainable code. The library is very light weight and performant.
https://www.nuget.org/packages/NetWorkflow
https://github.com/Tmarndt1/NetWorkflow
I'm currently trying to convince management of the need to port one of our applications to .NET. The application has grown to be a bit of a monster in Access (backend in SQL), with 700 linked tables, 650 forms/subforms, 130 modules and 850 queries.
I pretty much know all the major benefits of doing this, but now need to look at how this can be achieved technically, so I can put a project plan together.
So, my plan was to convert the queries into stored procedures and/or views on the backend and re-write the forms in WPF or WinForms.
Now, the code is where I come unstuck. Is it possible to packaged up the code behind and modules into dlls and consume them whilst it is slowly ported to VB/C#?
What we can't be left with is half an application in VB/C# and half in Access, it must 'appear' to all be one application, even half way through the migration.
Thanks in advance.
EDIT: Just some more info about what we do and why we're looking at moving away from Access.
We are essentially an ISV and the Access application is our main product. This application has been developed over a period of 15 years, by many, many developers on an ad hoc basis. There is no documentation for this application.
We also have problems with getting branching in SCC to work properly, so we've currently got 4 or 5 code bases for the half a dozen clients we have. On top of that, all the testing we do is completely manual, which you can imagine is very labour intensive, and only scratches the surface of what really needs to be tested.
We're currently looking to expand, and have a number of sales leads that are in the final stages. I'm worried that with these new sales, we're going to be swamped with support and testing, and that this application is going to become even more entangled an buggy.
I'll also add to this the fact that we're just about to enter the spec phase of a brand new product, which is almost certainly going to be built in .NET. If we were to rewrite the Access application in .NET, then the people we use for that can go straight on to this new development. If we were to stay in Access, then we'd have to get some new Access people in, whom would have to be retrained once we start the new development.
So essentially it has come down to two choices, major refactoring work in Access to try and 'organise' the code a bit better, and those of you who have suggested culling parts are most probably right; I'm sure there are parts that are no longer used. However, I fear that if we stay in Access we still won't be able to build in effective testing and we still won't have proper SCC branching, which will lead to support continuing to be a nightmare, and any future developments on this product makings things worse. Either way there is a lot of work that we're about to embark on, which is either going to be done in Acces, or .NET.
I been involved in a lot of migration projects where one is converting from one platform to another. I've also seen spectacular cost overruns, and spectacular under estimations of how difficult these types of projects can be.
In some of the projects and platforms I've created and that I had built for about $25,000, the cost of replacing this application and rewriting it resulted in the other team of people taking over this project and the resulting cost was in excess of $750,000.
You're also making an assumption that the current system needs to be replaced. You MUST have clear in your mind what the actual goals of moving and replacing the current platform and software are. Simply rewriting something and moving the functionality over to another platform yields you very little, except spending a lot of money that really doesn't benefit your business of all (but hey those developers will take the money, if they convince you of the need of doing this)
You might want to take a read of this wonderful article by Joel on software (Joel by the way developed and created this forum stack overflow – and I was a moderator of some of his discussion forums for almost 10 years),. In this article, Joel warns and gives caution against out of the blue simply rewriting perfectly good software that does not rust or wear out.
Things You Should Never Do, Part I
by Joel Spolsky
They did. They did it by making the single worst strategic mistake that any software company can make:
They decided to rewrite the code from scratch.
Article here: http://www.joelonsoftware.com/articles/fog0000000069.html
Joel continues to note that in the past 10 years, that article still remains one of his more popular ones (and somewhat controversial)
It makes no sense to take a perfectly good running application that's been running great for 10 years and doing its job, and then simply rewrite it in another platform, especially if you don't have the Manpower and expertise and personnel available to maintain this new system. And this is especially more so, if the new systems not going to accomplish anything more than what the previous system was doing. In fact if you did have that Manpower, they'd likely already have STARTED converting this system already over time. I mean why all of sudden out of the blue did someone throw a light switch and all of a sudden realize that new developers need to be brought in to rewrite a system that's already been running fine?
I'll also point out having been in this business for a long time (both published and technical editor of access books) I also done migration projects from mainframe systems to desktops . And, I also done migration of desktop database systems to large mainframe systems .
I can only say that it is rare to see an application with that many tables. In fact this issue raises alarm bells right off the bat.
Because of such a large number of tables here, I have to think that there's likely either very many processes, and multiple applications cobble together here that represents this whole system . If this is not the case, then of course rewriting in .net does not make sense unless you address the un normalized nature of the system. The fact that the data is already in SQL server helps, but that just might mean that you had the horsepower and capacity and infrastructure to scale something that was poorly designed and the first place
A very big major portion of software flexibility comes from having properly normalized data models. The problem you have is that you have the data in SQL server, and it's very tempting to rewrite parts of the forms and functionality as .net forms, and continue to use the current existing data models. Unfortunately this put you in a bit of a rock and hard place, because you want to continue to use existing data, and start rewriting functionality in .net. However rewriting of functionality in .net without addressing the data models is a very bad idea.
In an ironic twist of fate, this is a catch 22, because likely if that system had really fantastic great designed data models, you might not even have the need to redesign and move this into .net. Access and SQL server can scale out to 100's of users with ease anway. And, access supports the use of class objects, and even source code control.
In other words keep in mind that people might be asking to rewrite this in .net because they believe the application will then magically have increased flexibility, and be able to be changed faster then that of their changing business rules. In fact often the opposite occurs, because access is a very RAD tool. This means that the frontline people can often make modifications due to business rules changing, faster than the IT Dept and their team of developers working away on the next great version of the application. And worse you don't want to saddle that IT Dept and those developers with a poor data model.
I mean, are you to now hire the IT department to build every single spreadsheet and excel for the people because are current business processes are not flexible enough? It would be wonderful if the IT Dept to go around to everybody's desk, hold their hand, and build the excel sheets CORRECTLY for everybody, but it's not practical in the real world. So in addition to taking access away from these people, you might as well take excel away from them also.
I am just pointing out that my spider sense suggests to me that the data models here are going to be a real challenge. Remember, I would always take poor code and great designed data models over the reverse (Great code, but terrible data models). The reason for this is with great data designs, then the code and applications practically write themselves. And with great data models, then the ease of which you can change for the ever changing business rules again favors great data designs over that of great code. You can also RE factor the code overtime WHEN you have good data models. So, with good data models you can move forms and functionality and the UI over into .net, and you can do this seamlessly and easily WHEN the existing data models make sense to keep.
Also it makes no sense to move to these new technologies and less you're going to introduce the possibility of introducing things like self serve web portals for the existing business processes. So, today we can now allow customers to manipulate and use some of that information that is currently locked up in the system. This might be simple as them checking the status of their orders instead of wasting valuable customer phone time. Or it might be something simple like how a major package company in Canada saved an estimated $10,000,000 in the first year of implementing their package tracking system. Or might be something as simple as allowing the customer to look up their account balance.
So right now in the marketplace, these self serve customer web portal systems allow customers to enter and use and get at their information instead of the calling up some employed within the organization who then turns around and launches the application and then manipulates the information for that customer while on the phone. Might just as well let the customer do this work! So from order status, to balances owing, to banking, or what ever it is, the real cherry model ticket today is to allow the customer to use at a cell serve web portal that represents all that valuable information that that internal application is creating .
As mentioned, you have to ask, is where is the Manpower and personnel going to come to build and maintain this application? Obviously the existing system with enormous numbers of forms and tables you are throwing out must somehow been created, and represents some significant investments of time and efforts . The key concept you have to ask, is where were those significant investments and resources coming from to build that existing application? Who is going to maintain the new system then? In other words you need to design the new system to reduce maintenance costs. (new versions of my software can reduce maintenance by as much as 10 or 15 hours per year for customer ).
At the end of the day, good software development and good designs are good designs. Using Access, or VB6, or vb.net don't matter if the system is meeting your business needs now.
I should also point out that the new version of access 2010, can create .web forms. They are XAML (zammel .net forms). I am pointing this out, because changing the front end skin from access to .net yields you VERY little unless the underlying data structures and designs are also modified to take advantages of new possible business processes that can be accomplished with new technologies (such as those cell serve web portals). Simply repaint the font ends with .net forms is really very much amounts to a waste of money in my humble opinion UNLESS other issues are being addressed such as the data model, or some type of web portal that'll improve flexibility here.
You have some great advice here already. Keep in mind this really comes down to what are the goals and reasons for these people desiring this software to be rewritten in .net. Those new goals and desires better not be based on the pretext of simply remaking the forms you have now into .net as that will really accomplish nothing at all, and will not improve their ability to address their changing business needs that the system they are currently using obviously had been doing in the past.
Good luck on this I don't think this is the kind of question that can be answered in a simple forum post, but at least you have lots to chew on here so you can get the ball rolling.
Instead of telling you how hard it is -- which I'm sure you already realize -- I'll try to toss out some hints:
1) Start by moving everything you can out of MS Access and into MS SQL. This means tables, stored procedures, views, etc. If you get this step right, your MS Access app will be a front-end for a real database, which is already a win.
2) Consider giving up before you start. Instead of porting everything, it might make more sense to recognize which features can just be left alone, while new ones get WCF or MVC front-ends.
3) It's tempting to port from VBA to VB.NET, on the theory that it's more similar, but I don't recommend this.
I'm working in department which is mainly responsible for replacing old Access applications with .NET solutions. In my company Access applications are used for simple scenarios to fulfill business needs of single employee or small group of employees.
Sometimes Access application grows, group of users grows or too many changes are needed. In that case department using that Access application can start a project to recreate application. When this begins we can be sure that new application will be far away from current Access application.
First the business analyst is assigned to the project. His responsibility is to map current solution and to discuss problems of the current solution and expectations for the new solution. I haven't seen a project where "customer" wanted only replacing of the current solution. Everytime the customer also wants some new features and extensions which were not possible in Access.
Business analyst creates some initial description of expected solution which is passed to architects. Architects decide which type of application will be build, what type of HW infrastructure will be needed and how the application will be connected to other systems (if needed). After this initial phase IT have big picture about application and about needed changes. Here some initial estimate is done so project can be planned and resources can be allocated. This estimate is boundary for the project. Than my team starts to do the job.
We are using agile approach so our customer (internal team) incrementally sees new features in the application. First we gather some initial set (backlog) of user stories (special form of requirements) and we estimate these user stories and we let the customer to prioritize them. We choose subset of user stories for iteration (usually 2-4 weeks). New user stories can be added to backlog any time but our selected user stories can't change during iteration. After the iteration we present customer working part of the software. Based on the working part customer can decide to change priorities in backlog or create new user stories. We repeat this approach until customer says stop or unit budget is consumed. The important point is that not all user stories have to be done. Project has been planned with some budget and some low priority user stories don't have to be overtake.
From the technical point of view it is project as any other except few differencies:
You have initial database and you always have to be sure that already implemented part in your new solution also have working migration of exisiting data.
You have existing UI. Users can like or can hate the UI. Make sure that you understand it so that you create UI which is not worse than existing one. I created applications where UI had to be completely different and I created applications where UI had to be exactly the same so that users didn't need additional training.
Try to add some new features so that new application is reasonable. It is always easier to explain needs for the new application if you can describe new needed features.
650 forms/subforms is large by any standards. That represents a major conversion project, and a 'slow' port will be a nightmare.
I would suggest developing a new .NET application 'spike' that contains the basic functionality that is absolutely required and then build upon it. At the same time, freeze the Access application from all but essential fixes.
There are a few tools that will convert MS Access forms to .NET, but they will likely fail on complex forms with sub-forms.
'Effortlessly' Convert Access Forms to VB Objects
So if a user is in Access and they take an action to open a form that happens to be a different executable written in C#, won't it 'appear' to be the same application?
There has to be a user group that would love a separate application that only has the 5-10 forms they use.
Getting rid of tables/forms/features that are not used is an increase in functionality. I don't know the level of documentation for this app, but start there. When users find out they have to document areas of an application and justify the need for parts they don't use, they'll volunteer to have it removed.
I am currently working on an application in WPF/C# for personal use. I am not a "classically trained" programmer. Simply a hobbyist that likes to code in his spare time. Is there any accepted approach to the progression of application development? I. E.; Make it work, add fault tolerance, create a gui, then performance optimization. Or maybe should I design the entire GUI first? Basically I am going to start a new project soon and would like to have some sort of "every program needs this" checklist.
There's really no "every program needs this" list, because there's absolutely nothing that every program needs.
Some advice, though: don't "make it work", then "add fault tolerance". Defensive programming and accounting for errors should be a continuing part of development. It's much simpler (and usually more effective) if you account for errors and unexpected input when you're writing a piece of code rather than after it's done.
As far as whether or not to make the GUI first, answer this question: is the most important aspect of the program what it does or what it looks like? That's a serious question that, honestly, can vary from application to application (though it's usually the former that's more important).
If functionality is more important, model your information in code and get some basic "business logic" (a term of art that represents the non-visual logic in the application that carries out the rules and operations that are fundamental to the purpose of the program) in place, then create a GUI that interacts well with it.
If the GUI is genuinely more important, create it first and model data objects and business logic around the GUI.
I would advise you to peruse this Wikipedia article. It's pretty heady (as most technical Wikipedia articles are), but it provides some good links and will give you a rough idea of how the progression of software development and maintenance moves in the "real world".
I'd say first of all, as simple as it sounds, write (either on paper or in your head) what the app is actually going to do. (Most businesses can't even do this part!)
Knowing that, sketch on paper what you think the screens will look like.
For example, say you're writing a home budget program and you want a drop down for accounts, and a grid for transactions, etc. Knowing what the GUI looks like will help tremendously.
Then fill in the details.
If this is something you're doing as a hobby, you can choose any development methodology you want.
Typically, for commercial development, there's some form of prototype made first (for WPF, SketchFlow with Blend is fantastic for this). This is typically required since you most often need to "sell" the concept, either to a client, management, etc.
However, if you're doing this on your own, you have the freedom to do things any way you wish. One comment, however. I would, personally, not thing of "fault tolerance" as separate from "making it work", however, since "fault tolerance" comes with testing, and unit testing as you go is much more effective...
I try and take a look at my audience for the application. Then I'll create some use cases where I try and figure out what and why my application should even be built. Sometimes evaluating what you're writing and why you stumble across a lot of ideas that might shape the way you architect the application.
From there you can do as others suggest and put more importance on areas of focus.
This is a huge topic that will probably garner a lot of opinions, so I'll chime in with this: Test-driven development. Try reading about it, and make the effort to incorporate it into your development strategy as soon as possible. TDD is the one thing I wish I had known when I started writing software years ago. It makes a world of difference in the quality of the stuff you put out.
What has worked for me in the past is the following (high level) sequence:
Understand and record the requirements (who, what and when) - use-cases, feature list, flowcharts, service level requirements - whatever works for you (I am partial to use-cases though).
Understand and record the design (how) - class diagrams, sequence diagrams, screen designs, API specifications - start high level and drill down (could start development before complete drill down)
Implement - start with API/stubs and unit tests, then fill in the code, update unit tests and execute unit tests as you go.
System test - test the components operating together and fix defects. Don't forget about performance testing (check that you have met your service level requirements from step 1).
Package, deploy and enjoy.
The development lifecycle varies a lot. Variance is due to project size, team size, timeline, etc...
For small hobby projects, I usually follow this approach:
Specs: this could be a half page typed about what I think I'm doing, and maybe some diagrams to make it more clear.
Plan: Usually a document where I outline what I think major milestones will be, such as "complete proof of concept", "basic gui", "system logging", "successful CRUD operations"
Code: Try to meet the first milestone in 2., then possibly re-evaluate 2. Continue until project is done, or I get bored / distracted by something else (usually shinier than whatever I'm working).
This last step may also involve sub-steps such as data modelling (if it's a database app) or graphics design for icons (if the GUI needs fancy icons).