We have a base product that has bespoke development for each client that extends and overwrites the functionality of the base.
We also have a custom framework that the base product and sits on top of.
We use inherited forms to override the base functionality and to date all the forms and classes have been lumped in the same projects i.e. UI, Data, Business...
We need to clean up the code base now to allow multiple client project to run off the base product at once and I was looking for advice around the following areas:
Ways of organising the solution to fit with the above requirements, the number of projects in the solution is quite large and we want to reduce this to increase developer productivity, we are think of making the Framework DLL references instead of project references
Are there any build and deployment tricks we are missing, we currently have a half automated build and release process
What is the best way to manage versioning
Any best practices for product development
I personally strongly believe that highly modular architecture will fit here nicely: core application should provide basic/common services, and all customer-specific functionality should be implemented as plug-ins (think MEF). Hence, several thoughts:
I'd go for one solution for core application plus additional solution for each and every customer.
One-step build is a must. Just invest some time in writing a handful of MSBuild scripts: this will pay off tenfold.
See APR's Version Numbering for inspiration.
Too broad a question.
I can give you an advice on your first question and maybe a little of the forth : If i were you I would go with a framework DLL solution that could easily be managed and futher developed by a team and different solutions for each subsequest project. However, the framework solution would have to be propely developed, with extra care to one design principle: Open/closed principle [1] so future development of the framework does not break the existing implementations.
[1] http://en.wikipedia.org/wiki/Open/closed_principle
Related
I work in a .net c# application which contains 2 solutions for client and server. In server side there are 80+ projects that have been used to separate following Architectural layers,
Infrastructure Layer
Integration Layer (External Systems)
Domain Layer
Repository Layer
Manager Layer
Service Layer
In addition, almost every layer has test project.Now, the build time of the solution takes 2 to 3 minutes, and many developers (including me :)) feel we need to tackle this problem.
Therefore,proposed solution was to reduce the number of projects by merging the projects.In my view, it is probably a good solution to minimize the build time and we could achieve what we want.
Proposed solution is that we merge our projects into 3 areas, such as one library for production code, one library for test code, and one for deployment projects (WCF host ,etc) and logically divided layers in same project by separating the namespaces.
However, my concerns are
Could these separation good for the maintainability ? providing that more that hundread of classes for each namespace appox.
If we have common functionality such as helpers, where are we put those ?
Is there any other way to layering the solution ?
I guess you should split your solutions in logical layers.
As part of where do you put the helpers. Make a solution for it, on one of the lowest levels.
EXAMPLE
Software for a farm. You'll need to keep track of your animals, vegetables. You need a module for feeding the animals and one for Selling the animals and vegetables to the consumer market.
This could be splitted in a the following solutions
Back-end
Sell Module: Everyting for selling your products
Buy Module: Buying seeds, food for your animals, other products, ...
Sheduler Module: Trigger events for sow seeds, harvesting, ...
Prediction Module: Predicting harvests quantity's by the weather, and market prices, ...
...
Each of these back-end modules, can have it's own Data Access Layer, DTO, WCF Services, ...
This solution will only contain Business Logic, Data Access, ... . And there can be multiple front-end solution connecting to these back-end solutions.
Front-end
ASP.NET MVC Application: Webshop for selling to a consumer
WPF Application: Approving sells
Other WPF Application: Buying things.
Mobile application: Getting the events to your phone or something.
(Another option is to connect 2 or more backend solutions into 1 front-end solution)
...
This is a BIG change for your project and this will have an impact. Make sure you think this true, if you wan't to change it.
Multiple solutions will INCREASE your overall Build Time and it's important to have a nightly build so every developer can always work on the latest binaries, without having to build all the solutions on his local machine.
Note you can still use your layers in the different solutions:
Infrastructure Layer
Integration Layer (External Systems)
Domain Layer
Repository Layer
Manager Layer
Service Layer
To make this work all together and don't get messed up with binaries. You can map a drive I.E. X: where you have a folder binaries, where you have a folder for each solution. where each solutions copy's the assemblies on the post build event. (Script this, so it works on every machine)
If you have a good network infrastucture, you can also copy it on a server. So when you build all solutions for example in TFS, it can copy it to a location all developers can access.
If you build in TFS make sure your build order is correct, first the lowest layer, last the highest layer.
But as you split up your solution, in solutions you'll probably don't need them in every solution.
I recently read an article about Onion Architecture, maybe you can have a look at that too. (It's specific for ASP.NET MVC).
You can also have a look into CQRS.
Why 80+ projects while you only have 6 layers in your application ?
You might answer that they cover a large number of functional areas, but do you need all these functional areas in one solution in the first place ?
I'd recommend reflecting architectural divisions with projects and functional divisions with solutions. Different solutions can reuse the same projects. This way you'll have one project for each reusable architectural layer and as many Domain projects as there are functional areas.
I definitely wouldn't merge the projects... I think you'll quickly end up with spaghetti code in each layer as the developers take shortcuts (whether they mean to or not) that they shouldn't be taking.
I'd be more inclined to separate the layers out into separate solutions... and use binary references instead of project references across the tiers. This can play havoc with branching though, be careful.
I've seen build times drop by making the projects build to a common place - apparently this can prevent VS rebuilding projects when it doesn't need to - but I don't know if this is true or not.
Some ideas here: http://blogs.microsoft.co.il/blogs/arik/archive/2011/05/17/speed-up-visual-studio-builds.aspx
Finally.... is the three minutes for a full build or just to unit test one project? Focus on whichever is the biggest issue. If unit testing is taking a long time, you've got a problem with dependencies. If the full solution is taking a long time, get a build server and focus on bringing your unit test development time down.
Hope that helps
A low impact way I've dealt with a problem like that in the past is to create a series of solution files that include just one of the projects and its test project (and perhaps the project's dependencies). Then, get yourself a tool like NCrunch and do most of your coding in these solutions, probably using TDD. This will give you lightning fast feedback loops and is decidedly in the spirit of the layered, decoupled approach. When I've done this in the past, I find that I only actually run the entire application a few times a day, max, and I rely heavily on red-green-refactor, which is nice anyway.
If you want, you don't even have to source control these little solution files -- developers can create their own and they can be borderline throw-away.
Of course, this is by no means a panacea and won't address the problem of long compile times when you want to run the application, but it can definitely help simultaneously cut down on feedback time while promoting good design/development practice and it has the advantage of being extremely low risk and fast to setup.
I would like to ask you what experience you have with developing and deploying one application that in general has some standard features, but the application can also have customer specific features.
For example:
Customer 1 have the standard features but also want a search function.
Customer 2 have the standard features only.
Customer 3 have the standard features and also want an employee calendar.
How would you solve this?
Would you have one project where you deploy all the application from and then have some kind of config file to determind which features are avaliable in the specific application?
Would you have one project for each customer? This is how I'm doing it now, but the problem here is that if there are bugs that need to be fixed in the standard features I have to fix them in every project.
Any other suggestions are very welcome.
The application is developed in Delphi and C#.
My company solves that problem by giving all customers all features. This keeps development simpler and allows us to spend more time working on improving the product and not have to spend time dealing with the complexities of optional features.
We sometimes meet mild resistance from clients who want a cheaper version with less functionality but that's never been a sales problem.
On the other hand if you sell clients cheaper less functional versions, they are liable to try to get away with these cheaper versions. This can then lead to them not liking the software as much they should because they bought the cheap crippled version. I strongly believe in getting the best product possible to the user.
This advice may not be appropriate to your personal situation, but you did say that any opinions would be welcome.
One version per customer is not a good idea, IMHO. It will stuck your sales one day or later.
Better let all features be released to all customers, i.e. just maintain one software, but locked by a password, for instance. You can give a unique licence number at the software installation in order to identify the customer (put its name in the licence), then compute some passwords according to this licence number, to unlock some features, on request - when paid. This password can be easily automated via a web site, with minimal cost for you.
Or you can also let all functions available for testing, but lock the printing or the saving - just to let the customer think about spending some money to have this "nice added feature".
Sometimes, having all features tend to create a "Gasworks" application. You'll probably need a separate setup application, to customize the application to your customer's needs. Worth thinking about this architecture.
Even with a revision control, multiple versions are a nightmare to maintain: for instance, just back-port all hotfixes to a previous version takes a lot of time. If you don't have to (because of regulatory purpose / certified versions e.g.), don't "branch" your software.
No definitely do not have one project per customer, you could have one solution per customer where you agregate all projects given setup need.
Just to give you in an alternative to plugin architecture, which is right way to go, but also usualy fairly complex.
Option1.
Put common functionality in main project (Core)
Additional stuff like calendar put in separated DLL projects ( one per functionality)
Create VS SOLUTIONS, where you agregate all projects for specific setup + Core. So customer1 will have Customer1Silution with Core and all additional projects he need, customer2 its solution with Core and its additional stuff.
Option2.
Have one big setup for every one and based on its configuration/license enable/ disable access to user to a additional functionality.
Depends on your resources like time, experience, people you work with, clients , you can chose an option more appropriate to you.
1 plugin based: may be the best one but it, complex and it will take a time you become familiar with it, if you never did before something similiar.
Option 1 easy and fast, but if the clients quantity and configyration become defer you will jump into scale problems.
Option 2 is an average between those two, but keep an eye on your setup dimension.
Considering tha fact that you refer projects and nit DLLs in your solutions, if you fix a problem in Core in one solution it will affect also all other solutions.
you have several options:
put the "standard features" into separate module(s) which can be used/linked by the other versions
use a "plugin-architecture" to load the optional features dynamically
In addition to what others have said there is another option: Conditional defines.
With conditional compiling you can wrap feature specific code with IFDEFS (IFDEF EmployeeCalendar, IFDEF SearchFunction...). Then for each client you copy the project file only and set the conditional defines according to features you want to include.
If a client wants/pays for additional feature you just add it to Conditional defines in that client's project file.
This is similar to the modules approach (BPL/DLL) but avoids the added cost of having to deploy/manage extra files. The drawback is that the feature set is fixed at compile time.
With BPL/DLL you could dynamically load additional modules at run time, but if that is not important in your case, then Conditional defines might be a good choice.
Of course if your features are not easily separable then you can end up with a lot of IFDEFs in the code, but then your problem is clear separation of features, and it would be the problem with modules too.
We are in a situation whereby we have 4 developers with a bit of free time on our hands (talking about 3-4 weeks).
Across our code base, for different projects, there are a number of framework-y type of code that is re-written for every new project that we start. Since we have some free time on our hands, I'm in the process of creating a "standard" set of libraries that all projects can re-use, such as:
Caching
Logging
Although these 2 above would rely on libraries such as Enterprise Library, each new project would write its own wrappers around it, etc, so we're consolidating all these code.
I'm looking for suggestions on the standard libraries that you built in-house that is shared across many projects.
To give you some context, we build LOB internal apps and public facing websites - i.e. we are not a software house selling shrink-wrap, so we don't need stuff like a licensing module.
Any thoughts would be much appreciated - our developers are yearning to write some code, and I would very much love to give them something to do that would benefit the organization in the long run.
Cheers
Unit Testing Infrastructure - can you easily run all your unit tests? do you have unit tests?
Build Process - can you build/deploy an app from scratch, with only 1 or 2 commands?
Some of the major things we do:
Logging (with some wrappers around TraceSource)
Serialization wrappers (so you can serialize/deserialize in one line of code)
Compression (wrappers for the .NET functionality, to make it so you can do this in one line of code)
Encryption (same thing, wrappers for .NET Framework functionality, so the developer doesn't have to work in byte[]'s all the time)
Context - a class that walks the stack trace to bring back a data structure that has all the information about the current call (assembly, class, member, member type, file name, line number, etc)
etc, etc...
Hope that helps
ok, most importantly, don't reinvent the wheel!
Spend some time researching libraries which you can easily leverage:
For logging I highly recommend Log4Net.
For testing nUnit
For mocking, Rhino.
Also, take a look at Inversion of Control Containers, I recommend Castle Windsor.
For indexing I recommend Solr (on top of Lucene).
Next, write some wrappers:
These should be the entry point of you API (common library, but think of it as an API).
Focus on abstracting all the libraries you use internally in your API, so if you don't want to use Log4Net, or Castle Windsor anymore, you can by writing well structured abstractions and concentrating on loosely coupled design patterns.
Adopt Domain Driven Development:
Think of API(s) as Domains and modular abstractions that internally use other common APIs like you common Data Access library.
Suggestions:
I'd start with a super flexible general DAL library, that makes it super easy to access any type of data and multiple storage mediums.
I'd use Fluent nHibernate for the relational DB stuff, and I'd have all the method calls into the you data access implement LINQ, as it's a c# language feature.
using LINQ to query DBs, Indexes, files, xml etc.
Here is one thing that can keep all developers busy for a month:
Run your apps' unit tests in a profiler with code coverage (nUnit or VS Code Coverage).
Figure out which areas need more tests.
Write unit tests for those sub-systems.
Now, if the system was not written using TDD, chances are it'd be very monolithic and will require significant refactoring to introduce test surfaces. Hopefully, at the end of it you end up with a more modular, less tightly coupled. more testable system.
My attitude is that one should almost never write standard libraries. Instead, one should refactor existing, working code to remove duplication and improve ease of use and ease of testing.
The result will be very much like a "standard library", except that you will know that it works (you reran your unit tests after every change, right?), and you'll know that it will be used, since it was already being used. Otherwise, you run the risk of creating a wonderful standard library that isn't used and doesn't work when it is used.
A previous job encountered a little down time while the business sorted out what the next version should be. There were a few things we did that helped
Migrated from .net reoting to WCF
Searched for pain points in the code that all devs just hate to work with and refactor them
Introduce a good automated build system that would run unit tests and send out emails for failed builds. It would also package and place that version in a shared directory for the QA to pick up
Scripted the DB so that you can easily upgrade the database rather than being forced to take an out of date copy polluted with irrelevant data that other devs have been playing with.
Introduced proper bug tracking and triage process
Researched how we could migrate from winforms to wpf
Looked at CAB (composite application) or plugin frameworks so configuration would get simplier. (At that time setup and configuration was a tremendous amount of time)
Other things I would do now might be
Look at Postsharp to weave cross cutting concerns which would simplify logging, exception handling or anywhere code was repeated over and over again
Look at Automapper so that conversions from one type to another was driven by configuration rather than changing code in many places.
Look at education around TDD (if you dont do it) or BDD style unit tests.
Invest time in streamlining automated integration tests. (As this one is difficult to set up and configure manually it tends to get dropped of within SDLC)
Look at the viability on dev tools such as Resharper
HTH
I googled this a little but couldn't find a good result.
Right now I'm building a web site and I'm trying to make it as correct as possible from a design point of view from the beginning.
The problem I'm now facing is that when deciding to start with logging I needed a project to place this code in. As I could not find a suitable place in my currect projects I thought: hey, why not a logging class library?
Is there a general guideline on how many projects you should have? I know this would be a rather small project but it would be nice to entirely get it out of my way!
Any hints are appreciated :)
Absolutely you should have a logging library. And if you're going to make this as 'correct as possible from a design point of view' and your proect is less than trivial then you should definitely have some number of projects. The thing is, we have no idea what you're working on besides the fact that it's a web app. It's the biz domain that often determines how complex your solution has to be.
I'd go for a three-tier architecture for a small project.
This would include:
Application Layer
Business Layer
Data Layer
but if you want to add logging, it would be best to create another project. This would also help you so that if you want to add logging to another application, you can just include the logging project.
Either way would work, if you are concerned about project limit in a solutions. Don't be.
I myself would put it in a separate project or a utility project.
We have a solution with 200+ projects. The downside is long load time in Visual Studio. But past that the only issue is making sure you have enough RAM.
Also, MSBuild.exe has built-in support for SLN files, so look into using that instead of Visual Studio if you are doing automated builds.
"That which changes together should be packaged together", I forget where that guideline comes from (Code Complete maybe?).
In other words your assemblies (projects) should represent a coherent abstraction in the same way your classes/objects do at a lower level.
So yes, a separate logging project is the right way to go (although do check out log4net or Microsoft's logging block before you roll your own!)
Put a logging into a separate project is perfectly fine.However it really depends on the scope of your project.I normally setup my project like this
YourProject.Web(web project)
YourProject.Core(all the business logic)
YourProject.Web.Tests(Watin tests)
YourProject.Web.Core.UnitTests(Unit tests)
YourProject.Web.Core.IntegrationTests(Integration tests)
I suggest you to download some open source Project from asp.net to see how the project have been organized.
When/where do you decide to split a large Visual Studio project into smaller multiple projects? If it can be reusable? when project is too big? (but how big is too big?)
and When you do split the project, do you,
group by database tables
group by similar functionality
other..
Pros of many projects:
Easier to isolate code for unit testing. I like to isolate code that has a dependency on a big external server thing, for example code that talks to the SMTP server gets its own assembly, code that talks to the database gets it's own assembly, code that talks to the webserver, code that is pure business logic like validations.
Pros of few projects:
Visual studio goes faster
Some developers just don't get your vision
about dividing up responsibilities
and will start putting classes
everywhere, so you end up with the
pain of extra projects and the
benefits of putting everything into
one project.
Each project has a configuration and when you make a decision about project configuration, often you have to make the same chagne everywhere, such as setting or changing the strong name key
Pros of many Solutions
You hit the maximum project level later.
Only the stuff in your current solution gets compiled everytime you hit f5
If the project isn't expected to change in the life of your application, why re-compile it over and over? Call it done and move it to its own solution.
Cons of many Solutions
It's up to you to work out the dependencies between solutions and manually compile the dependencies first. This leads to complicated build scripts.
Projects should be cohesive. Logic should be related, and accomplishing a similar goal
This answer will depend on the size of the product you are supporting. In general we organize our projects along domain and logic. And we will divide those even further, the more you divide the more organize you must be, or you are going to hit the dreaded recursive dependency issue.
When I do choose to break up project it is when it grows to be too large or two areas are becoming too similar.
When complexity is rising I do not split by tables, i generally split functionality.
Re-usability is another excellent time to reduce lines of code, as well as introduce a new project. However be careful how many "utility" libraries you introduce because they do have impact on readability/understandability.
I do not think there is a line in sand that says, if you hit 3k SLOC, you have too much. It all is contextual.
I always have several projects (and therefore a solution) , instead of one project with all of my source in it.
In some cases, it is unavoidable because you are using and open source library and want to be able to debug it. But more pragmatically, I typically have my applications provide functionality via plugins. This allows me to change the behavior or offer a user-selectable behavior at runtime. In the non-plugin case, it allows you to update one portion of your program without updating everything. There are also cases where you can provide the main apparently, and only download the modules / assemblies when you need them.
One other reason is that you can create smaller test apps to exercise an assembly, rather than building a very large solution and potentially requiring a user to execute several (and irrelevant) GUI operations before even reaching the part you want to test. And this isn't just a testing concern -- maybe you have less-savvy users in your organization that only want to be presented with the bits that concern them.
When the overall purpose of the project remains the same, but the number of classes is becoming large, I tend to create folders and namespaces to better group functionality within the project. Classes that are coupled to each-other tend to go in the same folder/namespace, so that if I need to understand a given class, the related classes are nearby in the Solution Explorer. I usually only create new projects if I realize that a particular piece of functionality is very different in purpose or if there is a common dependency between existing projects.
I usually wind up with a few relatively small Framework projects that define interfaces for loose coupling between other projects, with larger projects for the different types of concrete functionality. That's always at least one project for the UI and one project for logic and data (often split into two projects if the data layer becomes very large in its own right.)
I move code to a new project, if it has general functionality (theoretically) usable by other projects too. If the project is large, because it represents a complex problem, then namespaces provide a great way to bring order in the code. Here you can for example introduce a (sub-)namespaces for each SQL table, etc. etc.