I try to realize microservice-based project on ASP.NET Core (Web API).
So, I have an independent components which communicate between them and external world.
So, I have "connection points" between services - view model (input data) of one is an equvivalent for request/response for others and so on.
I think, there are some best practices for this case which allow me not to create tonnes of identically code, am I right?
Deeply, let's look at the situation, where I have, for example, data in the database, and microservice, which get (possible transform or wide a little) an information from DB and give it to an asker. Is it possible not to create duplicate code for storing and responsing information from database?
Thank you.
What are you talking about are models. Input models and output models (DTOs).
If your projects are part of the same solution, then you can probably have a shared project or class library, to reuse your models.
If not, create a NuGet package, distribute it via your own feed and use it in all the projects that require it.
In order for this to work, you need to keep this project very simple. It should not have any dependencies preferably, so you can reference it without unintended consequences. If you keep it very simple then it can work well.
Mostly it depends on your use case, overall intent and solution architecture.
Micro services are meant to be autonomous from a development and deployment point of view and they shouldn't know about themselves or know as little as possible. The more they know about other micro services the higher is coupling. They should owns it's model and data needed for what they're created for (in order to meet their responsibility). You can achieve this using, for example, event based integration.
In this scenario I don't see a need for any code reuse. Every micro service will have different input and logic behind it. You should strive for this in your project.
If your micro services are too chatty (for example, they often need to ask other micro services for data), you probably made a mistake in their boundaries and you should consider design them again. Also you should avoid creating micro services which are just browsers for their databases.
Next thing to point out is DRY principle, and why this is not applicable to micro services world. In OOP world is common to use this principle. That's why most developers will try to use it in micro services world. But if you would try to apply this to micro services you'll end up with high coupling and you won't be able to develop them truly independently. Code reuse and data redundancy is not so bad as you probably think.
So to wrap up. As I said at the beginning it depends. If your "micro services" are part of one solution and you're for example referencing them in code, you can't name them micro services actually and you can use solution like Andrei told. But if they are not and you really care about their independence (and you're following what I mentioned above) you should not share code among different micro services, and there won't be need to actually. But if different micro services really use same code (even if they're well designed), don't be afraid and just reuse same code. You'll see that it will pay off.
Micro services are not silver bullet for every need and you should be aware of it. As a further reference I recommend you this free book.
Related
My team and I are looking for evidence to support either a multi library approach for like functionality, or condensing all of this functionality within a single service layer. It's important to note that this is going to be sitting behind a web api and either approach is valid, but we need to decide which holds more benefit. To illustrate the layer(s) we are looking at the following is what we'll have:
Solution
WebAPI
Services ---- This is what we're looking at
DataAccess
Bear in mind that if we did use the multi library approach we would still have a Services project, but it would be much thinner and have more specific functionality. We are not planning to independently deploy these libraries, but have everything needing to reference them either in the same solution, or access them via the web api.
What the rest of us would propose is something like the following:
Solution
WebAPI
Services
Services.Geography ---
Services.Membership --- This is the alternative approach
Services.ProductDelivery ---
etc...
The benefits we see in the first option is having all of this code organized within a single library which allows for easier extraction of duplicate code, potentially unit testing, and perhaps some relief from the build process.
The benefits of option two are having a clear delineation in functionality between projects, having isolated code which is potentially portable should the need arise, and generally being able to independently work on and configure different facets of the application.
The drawbacks we see in option one are that the Service layer now becomes responsible for each facet of the application, which bloats that library and in my opinion sort of violates Single Responsibility. We realize that rule is not as applicable to libraries so much as it is methods and classes, but it still seems like there are other benefits to be had by separating functionality. There's also the potential to mistakenly place code somewhere it doesn't belong, or use classes available to the entire project where they may not apply.
The drawbacks in option two are an obvious increase in overhead on project builds, working in configs (even though this may be desirable) and potentially cluttering the solution with more projects than necessary. I think we'd plan to consolidate like functionality into single projects (ie, we might build multiple implementations of ProductDelivery within that project to be able to switch between them or use different ones for different reasons).
We realize all of our business rules can be accomplished with either approach, we just have reached an impasse in deciding which approach is better practice.
So the question is, which of these two approaches is better practice?
I have 2 things that can make me think to chose the first option:
Your services use only one data layer library.
Your services are really short (something like implementing just the CRUD)
Split in libraries, a class that count few lines in an entire library, can be awkward. Unless, you know it will grow a lot after (in several classes, of course).
If not, I would say the option 2, is better because:
it's easier to replace a part of the service like that. Change the library that you want, and it's done.
it should be more abstracted, if you want to avoid strong coupling between each library
it should be easier to test a specific part of it.
it should be more configurable, and you can configure all of them in the project that references all of them (Even, if one or several libraries doesn't change a lot of things).
it should be less like a god library
it might be more exportable for some others projects, depending on how specialized your libraries are.
And I disagree, for these points:
a single library which allows for easier extraction of duplicate code
If you are careful, your duplicate code, can be extracted in a parent library, common to all the others. So all your duplicate code, should be automatically extracted (Except, if there is a lack of communication or people prefer to copy/paste code. But, one library would not change that. It might even be harder to find where the code already exists).
potentially unit testing
Why several libraries have to be more difficult to test ?
If you have several libraries, you will have to make them more abstracted, to allow the change. And then, your testing should be easy.
and perhaps some relief from the build process.
Why ? If all your libraries are well named, where would be your problems ?
Deploying a dll or several dlls, shouldn't be that hard.
If it's about the configuration, one library or more, you will still have the same configuration to make, not necessarily more (but probably a bit more).
I also disagree for the single responsability doesn't apply to libraries. It is.
Each library, should be responsible for one business, not all of them. If you finish with a set of libraries, it can become a framework. Even, for a framework, you will finish to have a single responsability but much more general, than the responsibility of methods, classes, libraries, etc...
But you might want an opinion from a more advanced architect/developer than me.
If someone disagree with me, don't hesitate to comment my answer. I would be happy to learn from your knowledge
With the comments from my first answer in mind.
The current plan is to have a single data layer. Many of our potential
other libraries would be third party api wrappers that don't
necessarily need to interact with the database. Those that do could
potentially have their own data layer which may or may not interact
with the same database or an independent database. I think doing that
makes them self contained and able to exist without the rest of the
solution. Still not totally sure if this is the approach we want to
take yet though.
Dependency injection ?
StructureMap as our IoC dependency resolver
You will end up with several libraries, unless, all your libraries you use, have to use themselves together.
You will either have your services becoming kind of proxies for the third party libraries, or your services will use proxies for the third party libraries.
But anyway, the proxy parts, should not be together in same library. It would be harder to change the third party library, if you do that.
If you chose the solution where your services use proxies for the third party libraries. You can inject these proxies into your services easily, thanks to the dependency injection.
If you change of third party library, change the proxy implementation and change the injection, and it's done.
But, if you chose to make your services the proxies. It's almost the same, but you have one layer less. And your service implementations have to be exported in different libraries. You will also have to be more careful when changing your service, because you will endup breaking things elsewhere in your app.
For that last point, having a proxy layer used by your services, sounds better to me, at the moment.
I'm still thinking. It will have more edits to come I imagine
The reason for asking this question is that I've been wondering on how to stitch all these different concepts together. There are many examples and discussions on i.e. DDD, Dependency Injection, CQRS, SOA, MVC but not so many examples on how to put them all together in a flexible way.
My goal:
Develop modules that with little or no modification can stand on their own
Changing or reworking the UI should be as easy as possible (i.e. the UI should do as little as possible, and be "stupid"
Use documented patterns and principles
To make it easier to ask a concrete question, the main arcitecture now looks like this:
The example shows how to add a note to an employee. Employee Management is one bounded context. Employee has several properties, among those an ICollection<Note>.
The bound context is in my understanding the logic place to seperate code. Each BC is a module. Most of the time I find each of them can warrant their own UI if needed (i.e. some modules might be made available for Windows phone).
The Domain holds all business logic.
The infrastructure holds repository implementation, and services to send mail, save files and utilities that does not belong in the domain. I'm thinking of making some of the common service feautures that I have to use in several domains (like send e-mail) as a sort of an API that I could reference to save some code implementing the same things across several BC's.
The query layer holds all Querys except GetById that I need in the repository to fetch an object. The query layer can query other persistence instances, and will probably need to change some for each UI.
The Wcf or Web Api is kind of my Application layer, it might belong in infrastrucure and not on the outside. This service also sets up the dependencies, so all UI need to do is to ask for information and send commands.
The process starts with the blue arrows. Read the model since that has most of the information.
In step 1 the EmployeeDto in this example is just some of employee properties to show the user information about the employee they need to make a note on (like a note about new experience or something like that).
So, the questions are:
Does implementing a layered arcitecture like this really involve so much mapping, or have I missed something?
Is it recommended (or even smart) to use a Wcf service to run the main logic like this (it practically is my Application Service)
Are there alternatives to Wcf without having my domain objects in my UI layer?
Is there anything wrong with this implementation. Any fall pits to look out for?
Do you have any good examples to recommend looking at that can help me to understand how all these concepts are supposed to work together.
Update:
I've read through most of the articles now (quite a bit of reading) except for the paid book (requires a bit more time to do). All of them are very good pointers, and the way of thinking of the Wcf of more as an adapter seems to be a good answer to question 2. JGauffins work on his framework is also very interesting if I'm planning to go the that route.
However, as mentioned in some of the comments beneath I feel some of the examples tends towards recommending or implementing event and/or command sourcing, message buses and so on. To me it is overkill to plan for that level of scaling right now. As many business applications this is a "large" (in terms of an internal application, think max a few thousand) number of users working on a large set of data, not a highly collaborative domain in the sense of needing to implement event and command queues often assosiated with CQRS to cope with that.
Based on the answers below, the approach I'll start with will be based on the model above and the answers like this:
I'll just have to cope with mapping. Thoe pros outweighs the cons.
I'll pull application services back to the infrastructure and
consider Wcf as an "adapter"
I'll use command objects and send to application service. Not
polluting my domain with domain objects.
To keep complexity down I try to manage without event/command
sourcing, message buses etc for now.
In addition I just wanted to link to this blog post by Udi Dahan about CQRS, I think things like this keeps complexity down unless they are really needed.
There is a trade-off between mapping and layers. One reason certain mappings exist is because appropriate abstractions aren't available or feasible. As a result, it is often easier to just explicitly map between layers than trying to implement a framework that infers the mappings, but I digress; this hinges on a philosophical discussion of the issue.
The WCF or WebAPI service should be very thin. Think of it as an adapter in a hexagonal architecture. It should delegate everything to an application service. There is conflation of the term service which causes confusion. Overall, the goal of WCF or WebAPI is to "adapt" your domain to a specific technology such as HTTP. WCF can be thought of as implementing an open host service in DDD lingo.
You mentioned WebAPI which is an alternative if you want HTTP. Most importantly, be aware of the role of this adapting layer. As you state, it is best to have the UI depend on DTOs and generally the contract of a service implemented with WCF or WebAPI or anything else. This keeps things simple and allows you to vary implementation of your domain without affecting consumers of open host services.
You should always be on the lookout for needless complexity. Layering is a trade-off and sometimes it can be overkill. For example, in an app that is primarily CRUD, there is no need to layer this much. Also, as stated above, don't think of WCF services as being application services. Instead, think of them as adapters between a transport technology and application services. In turn, think of application services as being a facade over you domain, regardless of whether your domain is implemented with DDD or a transaction script approach.
What really helped me understand is the referenced article on the hexagonal architecture. This way, you can view your domain as being at the core and you layer things around it, adapting your domain to infrastructure and services. What you have seems to already follow these principles. A great, in-depth resource for all of this is Implementing Domain-Driven Design by Vaughn Vernon, specifically the chapter on architecture.
Does implementing a layered architecture like this really involve so much mapping, or have I missed something?
Yes. The thing is that it's not the same object. It's different representations of the same object, but specialized for each use case. A view model contains logic to update the GUI, a DTO is specialized for transfer (might get normalized to ease transfer). etc. etc. They might look the same, but they really aren't.
You could of course try to put all adaptations into a single class, but that would not be very fun to work with when your application grows.
Is it recommended (or even smart) to use a Wcf service to run the main logic like this (it practically is my Application Service)
You need some kind of networking layer. I wouldn't let all client applications touch my database. It would create a maintenance nightmare if you mess with the database schema (if some of the clients still run the old version).
By using a server it's much easier to maintain version differences.
Do note the a WCF service definition should be treated as constant once being used. Any changes should be defined in a new interface (for instance MyService2).
Are there alternatives to Wcf without having my domain objects in my UI layer?
You could take a look at my framework. Start post: http://blog.gauffin.org/2012/10/writing-decoupled-and-scalable-applications-2/
Is there anything wrong with this implementation.
Not that I can see. Looks like you have a pretty good grasp of the concepts and how they should be used.
Any fall pits to look out for?
Don't try to be lazy with the queries and commands. Don't make them a bit more generic to fit several use cases. It will come back and bite you when the application grows. Smaller classes is easier to maintain.
Do you have any good examples to recommend looking at that can help me to understand how all these concepts are supposed to work together.
The my linked blog post and all other articles in that series.
Can you please provide me with some tips/guidelines when architecting, designing and implementing a .net framework application, with the requirements given below:
It will be an analytical tool which will retrieve data from files, sql databases and may be cubes. So data layer should be able to handle that. The middleware should be totally independent of the other layers so probably need an IoC container (which one would you recommend)
It will be deployed on the local intranet
The front layer might be WPF application or Silverlight in future (for now, I am concentrating on Silverlight but the point is that it will change)
It should be easy to customise and improve it in the future without changing much of the code as the framework will be deployed for many clients
I need a way to store the configuration information, which will be picked up by the application on application load events to set its feel and look.
I have two months to implement it and looking for as many tips as possible.
SoC for a start
break your application into several assemblies that use IoC (interfaces + implementations):
application model assembly - all other assemblies will reference this one because these classes will be used for inter-communication - they will mostly be just POCOs
presentation assembly - references app model and business services - this one is either WPF or Silverlight in any case use MVVM to make your testing life easier
business services assembly - references app model and data repositories assembly
data repositories - these define repositories that actually get data from the stores
Then I'd create three additional ones:
file data providers
database providers
cube providers
Data repositories would reference all three and use them to provide necessary data.
If configuration becomes very complex with a lot of functionality then you should put it in a separate assembly as well and reference it by business services assembly.
Which MVVM library to use
Since you mentioned time I suppose you'll have hard time catching your deadline. When using MVVM (which I suggested to use) I also suggest you don't use a full blown PRISM (a.k.a. Composite Application Guidance from P&P) but rather go with MVVM Light Toolkit. It will take you less time to get on the bandwagon.
Code generation
In places where appropriate I suggest you use T4 to its full potential. I use it to import stored procedure calls to avoid using magic strings when calling stored procedures (and using their parameters). Check my blog post about it as well.
DAL technology/library
Don't write your own data access code using things like SqlConnection/SqlConnection functionality. There're many data access layer libraries/technologies today that you can use and not reinvent the wheel. If you know nHibernate, then use that. If you know EF, then use that. If you know anything else, use that. Anything that will provide/generate as much code for you as possible that is already tested and debugged.
So it all boils down to:
DRY + YAGNI
a.k.a. Don't repeat yourself and You ain't gonna need it = don't over-engineer you code.
Agile developers are supposed to be lazy
They should develop just as much as it's needed and no more! TDD implicitly provides this process by the red => green => refactor steps.
I would recommend using MVVM and Test Driven Development. The MVVM will give you good separation between the front and middleware, and the TDD will help control the chaos that comes with any nontrivial app development.
Have a look at the Composite Application Guidance from Microsoft's Patterns and Practices group, it may not match what you are doing exactly but will give you some good ideas.
From an architectural standpoint, I highly recommend taking a look at the Microsoft Application Architecture Guide. Since you are already using the Microsoft technology stack, I would consider using Microsoft Unity for IoC. You indicated that your presentation layer might use WPF or Silverlight, so take a look at using Windows Communication Foundation, as you will be somewhat constrained in Silverlight when it comes to communication with your data layer.
I'm developing an three layer ASP.NET application with C# and Visual Studio 2008 SP1. I'm using WebForms.
I'm wondering to convert that application to a Silverlight application. Maybe I can reuse a lot of code of ASP.NET layer.
What do you think about?
Assuming you have the typical presentation, business logic, and data layers, and also assuming that you have separated your code diligently into these layers, you should be able to replace your Web Forms with a Silverlight interface and leave your BL and DAL intact.
Real projects tend to be somewhat messy, however, making such a transition more difficult. If you're using SqlDataSource you might have problems.
Those are some good points #Andy, and to expand on what he said:
i'm doing that very same thing right now. Because i have a rather comprehensive business layer, i have been able to do a lot of work (a couple of weeks worth), and in that time i have only had to add one function to that business layer. This is important because it reduces the amount of testing required. It also makes any remaining testing easier as it is easier to compare the output of the old version of the application with the new version.
One pattern that really helped to achieve this was the facade pattern. I built a WCF layer that sits over top of the business layer, and by using the facade pattern i can return results that are more suitable for the new silverlight interface, without interfering with the business layer.
It is most likely though that your new UI will have a drastically different architecture than the ASP.NET version. You will be able to achieve a far cleaner separation between UI, code and data. Some of the ASP.NET code that i was quite proud of looks positively mangy next to the equivalent silverlight code. Be prepared to chop your old code up, and eliminate those business rules from the immediate code behind :)
If you're goal is simply to replicate the UI behaviour as delivered by ASP.NET then yes assuming good partitioning you could re-use quite a bit of code. You'd have ask why you would want to do that though.
On the other hand if the goal is to provide a much richer interactive experience to the user then its likely that you'll find even a well designed business layer just doesn't behave the way such a radically different UI needs it to.
Can anyone confirm the best way to integrate the repository pattern with webservices.... Well actually i have my repository patter working now in c#. I have 3 projects, DataAccess, Services and my presentation layer.
Problem is my presentation layer is a number of things... I have a ASP.NET MVC site, I have an WPF application and we are about to create another site + an external company needs access to our repository also.
Currently i have just added the services layer as reference to each of the sites... But is not the normal way to provide data access via web services? (WCF) - if this is the case will this break the services layer? or should i convert the services layer to a web service?
Anybody know what the PROS and CONS are of this, speed??
I think I understand your dilemma. If I understand correctly then your services layer consists of pure fabrications. http://en.wikipedia.org/wiki/GRASP_(Object_Oriented_Design).
If I assume correctly above, then your services layer should not be impacted at all by the introduction of WCF. WCF is essentially an additional presentation layer that provides interoperability, sitting between your UI presentation layer and any business logic layers. So your WCF services would then call your services layer, which may access repositories as needed.
WCF provides a high degree of interoperability so I think it is an excellent choice. I would use basicHttp bindings though, if you intend to interop with different programming languages as this is the most flexible. Don't worry about the speed. There are plenty of solutions out there to mitigate any bottlenecks that result due to WCF.
Good luck, and let me know if I can help in any other way.
Well first - not all callers have to use the same repository API; this is especially true of an external company.
WCF is interface based. This means that if you need to re-use some logic code, it is possible to use IoC/DI to inject WCF rather than a DAL (but using the same interface) - by using assembly sharing. It sounds like this is what you are doing. This works in many cases, but not all; fundamentally web-service based APIs often need to be designed differently in order to be optimal. It also isn't 100% pure from an SOA viewpoint, but it gets the job done, and allows more intelligent domain entities, so in an intranet (etc) scenario it is (IMO) perfectly reasonable.
An external caller would typically just use the wsdl/mex-based APIs (rather than assembly sharing), but anything is possible...
Maybe webservices are not the best way, if i have full access to the service assembly then i suppose it always better to assembly share the services layer with my applications.
My applications do similar things, but they all need to access the service layer - well the business logic and get back information...
In this case - its always preferable to use assembly sharing with the service layer rather than provide a WCF Web service using HTTP protocol or using TCP on wcf - for example?
Thanks again
Whether to share your Service/API assemblies with your client applications is fairly subjective. If you are a full Microsoft shop, and use .NET for your entire application stack, then I would say sharing the API is a great way to gain code reuse (you have to be careful how you design your API so you don't bleed domain concerns, like repositories, into your presentation.) If you don't have any plans to migrate your client applications to other platforms (i.e. you plan to stay on .NET for the foreseeable future), then I think its perfectly acceptable to share your Service/API assemblies (and even then, in a multi-platform client environment, sharing Service/API with .NET clients should still be acceptable.) There is always a trade off between the 'architecturally ideal' and the 'practical and achievable within budget'. You can spend a LOT of time, money, and effort trying to achieve the architecturally ideal, when the gap between that and the practical often isn't really that much. The choice NOT to share the API and essentially recreate it to maintain "correct" SOA, consuming only the contract, can actually increase work and introduce maintenance hassles that quite possibly are not worth it for your particular project at this particular time. Given that you are already generally 'service-oriented', if at a future point in time you need the benefit that contract-only consumption on the client can offer, then your already set to go there. But don't push too far too soon.
Given your needs, from what I have been able to glean from these posts so far, I think your on the right track from your services down too. A repository (a la Evans, DDD) is definitely a domain concern, and as such, you really shouldn't have to worry about it from the perspective of your presentation layer. You services are the gateway to your domain, which is the home of your business logic. Repositories are just a support facility that helps you achieve domain isolation from a data store (they are glorified collections really, and to be quite frank...they can be a bit of a pain in a dynamic and complex domain. Simple data mappers, (Fowler, PofEAA) are often a lot easier to deal with and less complex in the long run, and allow more adaptable behavior around your data retrieval logic to be centralized in your domain services.) Aside from heavy use of AJAX calls to REST Services, if you expose adequate Services/API around your domain, that is the only thing that your clients should have worry about. Wrap up all the rest of your business logic entirely within the confines of your domain, and keep your clients as light weight as possible and abstracted from concepts like 'Repository' or 'Data Mapper' and whatnot.
In my experience, the only non-service or API concept that needs to be shared across the Client-to-Domain boundary is Context...and it can be notoriously difficult to cross that boundary in a service-oriented application.