I am quite new when it comes to RabbitMQ and I am working on a POC to run a C# solution that's using RabbitMQ in a docker container.
I managed to write the docker-compose.yml, dockerfile and run RabbitMQ. However, my solution cannot reach the RabbitMQ host. I think I might be missing some configuration but I cannnot tell what.
docker-compose.yml
version: '3.4'
services:
rabbit-sender:
image: rabbit-sender
container_name: rabbit-sender
build:
context: ../SenderRabitMQ
dockerfile: debug.Dockerfile
env_file: common.env
networks:
- rabbitPoc
expose:
- "80"
rabbit-receiver:
image: rabbit-receiver
container_name: rabbit-receiver
build:
context: ../ReceiveRabitMQ
dockerfile: debug.Dockerfile
env_file: common.env
networks:
- rabbitPoc
expose:
- "80"
rabbitmq:
image: rabbitmq:3.7.15
hostname: rabbitmq
build:
context: rabbit
dockerfile: debug.Dockerfile
ports:
- "5672:5672"
- "15672:15672"
volumes:
- "./enabled_plugins:/etc/rabbitmq/enabled_plugins"
debug.Dockerfile
Install RabbitMQ
FROM ubuntu:14.04.1
CMD docker pull dockerfile/rabbitmq
CMD docker build -t="dockerfile/rabbitmq" github.com/dockerfile/rabbitmq
FROM dotnet-core-sdk-2.1-debug:latest AS build-env
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY SenderRabitMQ/SenderRabitMQ.csproj SenderRabitMQ/
RUN dotnet restore SenderRabitMQ/SenderRabitMQ.csproj
# Copy everything else and build
COPY ./ ./
RUN dotnet publish SenderRabitMQ/SenderRabitMQ.csproj -c Debug -o out --no-restore
# Build runtime image
FROM dotnet-core-aspnet-2.1-debug:latest
WORKDIR /app
COPY --from=build-env /app/SenderRabitMQ/out .
ENTRYPOINT ["dotnet", "SenderRabitMQ.dll"]
RUN command
docker run --hostname myrabbit rabbitmq:3
Connecting to RabbitMQ
var factory = new ConnectionFactory() { HostName = "myrabbit:5672" , DispatchConsumersAsync = true };
This is the error received when running the RabbitSender that's supposed to post a message to the queue.
rabbit-sender | Unhandled Exception:
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the
specified endpoints were reachable ---> System.AggregateException:
One or more errors occurred. (Connection failed) --->
RabbitMQ.Client.Exceptions.ConnectFailureException: Connection
failed --->
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException:
Connection refused 127.0.0.1:5672
Your docker compose sets the RabbitMQ service host name to be rabbitmq and not myrabbit (which is what you're trying to connect to). Try this instead:
var factory = new ConnectionFactory() { HostName = "rabbitmq", port = 5672 , DispatchConsumersAsync = true };
You also will need the Dockerfile rabbitmq section to be on the same network as the other services:
rabbitmq:
image: rabbitmq:3.7.15
hostname: rabbitmq
build:
context: rabbit
dockerfile: debug.Dockerfile
ports:
- "5672:5672"
- "15672:15672"
networks:
- rabbitPoc
volumes:
- "./enabled_plugins:/etc/rabbitmq/enabled_plugins"
Hope that helps!
You should use
HostName = "http://host.docker.internal:5672"
or
HostName = "host.docker.internal:5672"
instead of
HostName = "myrabbit:5672"
The reason is:
The host has a changing IP address (or none if you have no network
access). From 18.03 onwards our recommendation is to connect to the
special DNS name host.docker.internal, which resolves to the internal
IP address used by the host. This is for development purpose and will
not work in a production environment outside of Docker Desktop for
Windows.
https://docs.docker.com/docker-for-windows/networking/
In docker container,the place where you are making a connection to locally setup rabbitmq, you need to give the host as following
host.docker.internal
It will work.
Related
I am trying to run my application in docker while also creating a SQL Server.
It does not seem to connect to my SQL Server and I get the following error:
Unhandled exception. Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server: Could not open a connection to SQL Server)
This is how my docker-compose file looks like.
version: '3.7'
services:
musicapi:
build:
context: .
ports:
- 80:80
environment:
Authentication:Enabled: 'true'
restart: always
depends_on:
- testdb
testdb:
image: mcr.microsoft.com/mssql/server:2017-latest
ports:
- "1433:1433"
environment:
TZ: Europe/Amsterdam
ACCEPT_EULA: "y"
SA_PASSWORD: Password01
volumes:
- mssql_data:/var/opt/mssql/data
restart: always
volumes:
mssql_data:
And my Dockerfile:
FROM mcr.microsoft.com/dotnet/aspnet:6.0-alpine AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:6.0 as build
WORKDIR /src
COPY . .
RUN dotnet restore
RUN dotnet build -c Release
RUN dotnet publish -c Release -o ./out
FROM base AS final
WORKDIR /app
COPY --from=build /src/out .
# The following 2 lines are required for solving:
# "Globalization Invariant Mode is not supported" error.
RUN apk add --no-cache icu-libs
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ENTRYPOINT ["dotnet", "Test.Api.dll"]
My connection string looks like this:
"ConnectionStrings":
{
"DefaultConnection":
"Server=localhost,1433;Database=test.db;User ID=sa;Password=Password01;TrustServerCertificate=True;"
}
What am I doing wrong?
Each container is a separate network entity, so when your connection string specifies Server=localhost, that means that the database is in the same container as your application.
On the Docker network that docker-compose creates, the containers can talk to each other using the service names as the host names. In your case, the database container can be reached using the host name testdb.
So you connection string should be
Server=testdb,1433;Database=test.db;User ID=sa;Password=Password01;TrustServerCertificate=True;
I know this looks duplicated, but I checked all the others questions and none of them solved my problem.
So, this is my docker-compose yml file:
version: '3.8'
services:
#db:
# image: postgres
messageBroker:
image: rabbitmq:management
hostname: "messageBroker"
healthcheck:
test: rabbitmq-diagnostics -q ping
interval: 5s
timeout: 15s
retries: 3
networks:
- services-network
ports:
- "15672:15672"
- "5672:5672"
environment:
RABBITMQ_DEFAULT_USER: "admin"
RABBITMQ_DEFAULT_PASS: "password"
serviceDiscovery:
image: steeltoeoss/eureka-server
hostname: eureka-server
networks:
- services-network
ports:
- "8761:8761"
order-api:
image: ${DOCKER_REGISTRY-}orderapi
hostname: orderapi
environment:
- Eureka__Client__ServiceUrl=http://serviceDiscovery:8761/eureka/
- Eureka__Client__ShouldRegisterWithEureka=true
- Eureka__Client__ValidateCertificates=false
networks:
- services-network
depends_on:
- serviceDiscovery
build:
context: .
dockerfile: Services/Order/Dockerfile
links:
- "serviceDiscovery"
product-api:
image: ${DOCKER_REGISTRY-}productapi
hostname: productapi
restart: on-failure
environment:
- Eureka__Client__ServiceUrl=http://serviceDiscovery:8761/eureka/
- Eureka__Client__ShouldRegisterWithEureka=true
- Eureka__Client__ValidateCertificates=false
networks:
- services-network
depends_on:
messageBroker:
condition: service_healthy
serviceDiscovery:
condition: service_started
build:
context: .
dockerfile: Services/Products/Dockerfile
links:
- "serviceDiscovery"
- "messageBroker"
networks:
services-network:
this is my config file which I connect to RabbitMq:
using RabbitMQ.Client;
namespace MessageBroker;
public static class MessageBrokerConfig
{
public static IModel ChannelConfig()
{
var channel = new ConnectionFactory { Uri = new Uri("amqp://admin:password#messageBroker:5672") }
.CreateConnection()
.CreateModel();
return channel;
}
}
but when I run docker-compose up I still got the error:
product-api_1 | Unhandled exception. RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable
product-api_1 | ---> System.AggregateException: One or more errors occurred. (Connection failed)
product-api_1 | ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed
product-api_1 | ---> System.Net.Sockets.SocketException (111): Connection refused
And the product service can register inside the Service Discovery without a problem, and I followed almost the same steps.
And I know that the problem isn't the rabbitmq container taking time to be ready, because I can connect on my machine. And everytime the product service failed to launch, it restarts, but no matter how much time it takes, I still got this error. And the log of the messageBroker container shows it's healthy (and if wasn't, I would not be able to access through my machine ).
I don't have any other ideas, I'm on this problem 3 days alredy and I'm going crazy. I checked tutorials, followed the steps and nothig.
Solved, guys! Well, the configuration was correct. However, whenever I created a new container, it didn't build. Therefore, even if I changed the code, it still had the image of the first version of the code, with the "localhost" as the hostname. So the only thing I did was delete the image of the service and Docker created a new one with the correct code.
I see this obviously as a temporary solution, since it has to build everytime it runs/a new container is created. But this is another subject and I think it won't be hard to implement. Maybe with the arg --build in docker compose is enough. But I will only give attention to this later.
Your docker-compose file does not have the networking setup correctly. IMO you don't need the links. Here is a minimal docker-compose that worked for me. I removed the links and I removed the service discovery which isn't in play here for connectivity between rabbitClient and the broker.
version: '3.8'
services:
#db:
# image: postgres
messageBroker:
image: rabbitmq:management
hostname: "messageBroker"
healthcheck:
test: rabbitmq-diagnostics -q ping
interval: 5s
timeout: 15s
retries: 3
networks:
- services-network
ports:
- "15672:15672"
- "5672:5672"
environment:
RABBITMQ_DEFAULT_USER: "admin"
RABBITMQ_DEFAULT_PASS: "password"
product-api:
image: ${DOCKER-REGISTRY-}productapi
hostname: productapi
restart: on-failure
environment:
- Eureka__Client__ServiceUrl=http://serviceDiscovery:8761/eureka/
- Eureka__Client__ShouldRegisterWithEureka=true
- Eureka__Client__ValidateCertificates=false
networks:
- services-network
depends_on:
messageBroker:
condition: service_healthy
build:
context: client
dockerfile: Dockerfile
networks:
services-network:
On the docker-compose, in the network is missing something, put it like this:
networks:
services-network:
driver: bridge
I have an asp.net webApi application and using redis stack as the main database. I was able to connect to the database and perform operations normally till I made a docker image from my application and tried to run it on docker. The application cannot connect to the redis stack even that I have changed "localhost" to the "container name" and set it as an environment variable in my docker-compose file. Here is my code to connect and my docker-compose file for reference.
Connection to Redis Stack Code (Using Redis OM Package):
builder.Services.AddSingleton(new RedisConnectionProvider(builder.Configuration[Environment.GetEnvironmentVariable("REDIS_CONNECTION_STRING", EnvironmentVariableTarget.Process)]));
builder.Services.AddHostedService<IndexCreationService>();
Dockerfile
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["API/API.csproj", "API/"]
COPY ["Core/Core.csproj", "Core/"]
COPY ["Persistence/Persistence.csproj", "Persistence/"]
COPY ["Service/Service.csproj", "Service/"]
RUN dotnet restore "API/API.csproj"
COPY . .
WORKDIR "/src/API"
RUN dotnet build "API.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "API.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "API.dll"]
Docker-compose.yaml
version: '3.7'
services:
redis:
image: redis/redis-stack
container_name: redis
volumes:
- db-data:/data/redis
ports:
- "6379:6379"
- "8001:8001"
restart: unless-stopped
networks:
- course-network
CourseService:
image: dexam/course-service
container_name: course-service
volumes:
- course-service-data:/data/course_service
ports:
- "443:443"
environment:
- REDIS_CONNECTION_STRING=redis://redis:6379
restart: unless-stopped
depends_on:
- "redis"
links:
- "redis"
networks:
- course-network
volumes:
db-data:
course-service-data:
networks:
course-network:
driver: bridge
The error message I get:
Unhandled exception. StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). Error connecting right now. To allow this multiplexer to continue retrying until it's able to connect, use abortConnect=false in your connection string or AbortOnConnectFail=false; in your code. at StackExchange.Redis.ConnectionMultiplexer.ConnectImpl(ConfigurationOptions configuration, TextWriter log) in //src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1162
at StackExchange.Redis.ConnectionMultiplexer.Connect(ConfigurationOptions configuration, TextWriter log) in //src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1028
at Redis.OM.RedisConnectionProvider..ctor(String connectionString)
at Program.$(String[] args) in /src/API/Program.cs:line 17
Can anyone spot the problem?
It looks like there's a bit of confusion in whether you're pulling your connection string out of your environment or configuration:
builder.Services.AddSingleton(new RedisConnectionProvider(builder.Configuration[Environment.GetEnvironmentVariable("REDIS_CONNECTION_STRING", EnvironmentVariableTarget.Process)]));
This line doesn't really make sense, you are trying to pull a configuration variable from your appsettings file whose name is the VALUE of your REDIS_CONNECTION_STRING environment variable (which you've set in the docker-compose as redis://redis:6379) - this is most likely coming up as an empty string and hence, failing to parse-correctly and connect - the default behavior when Redis OM encounters an empty string in the config is to connect to redis://localhost:6379, since there's no redis instance there it will fail like the way you've shown.
You're looking for 1 of two things
You want to pull the connection string out of the environment variable, if you change that line to the following, given your docker-compose file, this ought to just work
builder.Services.AddSingleton(new RedisConnectionProvider(Environment.GetEnvironmentVariable("REDIS_CONNECTION_STRING", EnvironmentVariableTarget.Process)));
You really do want to pull the connection string out of the configuration, this is valid, but you'll want to make sure that you are setting the connection string in your configuration before you build the image:
builder.Services.AddSingleton(new RedisConnectionProvider(builder.Configuration["REDIS_CONNECTION_STRING"]));
Both methods are valid, but they won't play nice together which is why you're getting a bit of heartburn here.
I am using docker compose with a .net core service and RabbitMQ. I try to connect to rabbitMQ from a different service. I've seen many questions but none of them worked or had an answer.
I know you should not use 'localhost' to connect with another service but the name you gave in the docker-compose file.
I am really curious about the answer!
What I tried
Add depends on in the docker compose file
Add links in the docker compose file
Add restart: always in the docker compose file
Changed the connection link to : amqp://guest:guest#rabbitmq:5672
Changed the connection link to : amqp://guest:guest#rabbitmq/
Cleaned and rebuilt the solution many times
Changed the order in the docker compose file. RabbitMQ is now Nr.1, after that the other services.
Added container_name
Added hostname
Here is my docker compose file
version: '3.0'
services:
rabbitmq:
container_name: rabbitmq
hostname: rabbitmq
image: rabbitmq:3-management
ports:
- "7100:15672"
- "7101:5672"
volumes:
- rabbitmq:/rabbitmq
dbPosts:
image: mysql:latest
ports:
- 3307:3306
environment:
MYSQL_ROOT_PASSWORD: rootpassword
MYSQL_DATABASE: postsdb
volumes:
- dbpostdata:/var/lib/mysql
post-service:
depends_on:
- dbPosts
- rabbitmq
build:
context: .
dockerfile: post-service/Dockerfile
ports:
- "8081:80"
volumes:
dbpostdata:
rabbitmq:
How the connection is made :
if (_connection == null)
{
ConnectionFactory factory = new ConnectionFactory()
{
Uri = new Uri("amqp://guest:guest#rabbitmq:5672"),
AutomaticRecoveryEnabled = true
};
_connection = factory.CreateConnection();
}
But when I try to run docker-compose up, I receive the following error message:
Unhandled Exception: RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No such device or address
I fixed the issue. I got the wrong package version...
I have my simple app in C# that connect with postgreSQL.
I would like to create image with this app and just run with docker.
Everything is ok when I use:
$ docker build
$ docker run postgres
$ docker run my_app
Additionally, there is everything ok, when I use compose from application directory:
$ docker-compose build
$ docker-compose up
But is there any chance for use docker-compose for image that I built previous?
I would like to publish this image to my repo and somebody else from my team just download and run this image (app + database).
When I do compose-build and next compose run my_app I have exception during connecting to database:
dbug: Npgsql.NpgsqlConnection[3]
Opening connection to database 'POSTGRES_USER' on server 'tcp://postgres:5432'.
Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.AggregateException: One or more errors occurred. (No such device or address) ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No such device or address
My current docker-compose.yml file:
version: '2'
services:
web:
container_name: 'postgrescoreapp'
image: 'postgrescoreapp'
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/var/www/postgrescoreapp
ports:
- "5001:5001"
depends_on:
- "postgres"
networks:
- postgrescoreapp-network
postgres:
container_name: 'postgres'
image: postgres
environment:
POSTGRES_PASSWORD: password
networks:
- postgrescoreapp-network
networks:
postgrescoreapp-network:
driver: bridge
you should build the image with this name: (registryName:RegistryPort)/imagename:version
$ docker build -t myRegistry.example.com:5000/myApp:latest .
$ docker build -t myRegistry.example.com:5000/myDb:latest .
Now add these lines to the docker-compose file :
Myapp:
image: myRegistry.example.com:5000/myApp:latest
MyDb:
image: myRegistry.example.com:5000/myDb:latest
And then push it :
$ docker push myRegistry.example.com:5000/myApp:latest
$ docker push myRegistry.example.com:5000/myDb:latest
Your mate should now be able to pull it now
$ docker pull myRegistry.example.com:5000/myApp:latest
$ docker pull myRegistry.example.com:5000/myDb:latest
Yes, You can use previously created image from repository via docker compose.
Example:
version: '2'
services:
app:
image: farhad/my_app
ports:
- "80:80"
networks:
- testnetwork
postgres:
image: postgres:latest
networks:
- testnetwork
networks:
testnetwork:
external: true
Example Explaination:
I'm creating a container named app from my repository and another container named postgres via library postgres image.
Please note you need to create an push your custom image to repository first.
I'm using user-defined networks here, You need to create testnetwork before docker-compose up.