Develop Reference

Unable to load the service index for source error for a CodeArtifact feed using a Dockerfile

It's my first question here so let me know if I have to change something or add more information.
We’re using AWS CodeArtifact for storing our packages and when we try to build a Docker image from our Dockerfile it fails because it's unable to load the source during the restore process.
We have a web API in .Net we want to deploy using AWS Fargate. This project runs smoothly from Visual Studio 2022 using Docker but we can’t build the image from PowerShell after adding our packages from CodeArtifact.
Our approach to include the credentials in the build is to pass the NuGet.Config stored on the host using Buildkit.
This’s our Dockerfile:
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
ENV ASPNETCORE_URLS=http://+:49151
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY . .
WORKDIR /src
COPY ["Src/Presentation/Project.API/Project.API.csproj", "Src/Presentation/Project.API/"]
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
--mount=type=secret,id=nugetconfig \
dotnet restore "Src/Presentation/Project.API/Project.API.csproj" \
--configfile /run/secrets/nugetconfig
COPY . .
WORKDIR "/src/Src/Presentation/Project.API"
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
dotnet build "Project.API.csproj" -c Release -o /app/build \
--no-restore
FROM build AS publish
RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
dotnet publish "Project.API.csproj" -c Release -o /app/publish \
--no-restore
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Project.API.dll"]
Our script in PowerShell:
docker buildx build --secret id=nugetconfig,src=$HOME\AppData\Roaming\NuGet\NuGet.Config -f "Src\Presentation\Project.API\Dockerfile" -t my-dotnet-image .
Output:
#14 [build 6/9] RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages --mount=type=secret,id=nugetconfig dotnet restore "Src/Presentation/Project.API/Project.API.csproj"
--configfile /run/secrets/nugetconfig
#14 1.175 Determining projects to restore...
#14 2.504 /src/Src/Presentation/Project.API/Project.API.csproj : error NU1301: Unable to load the service index for source
https://domain-123456789012.d.codeartifact.us-east-2.amazonaws.com/nuget/repository/v3/index.json.
What are we missing here? Once we have the Dockerfile working we want to use CDK for deploying the Docker image alongside our infrastructure.
We’re using aws codeartifact login command to authenticate with the service. We work on windows with a Linux container

Building docker image with package from private nuget feed fails

I'm using the default Dockerfile generated by Visual Studio 2022 for my .NET Core 6 razor pages app. The app is using DevExpress ASP.NET Core Controls. I downloaded the necessary packages using the private Nuget feed URL supplied by the package vendor.
This is how the Dockerfile looks like:
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["NuGet.Config", "."]
COPY ["DMS.Monitor/DMS.Monitor.csproj", "DMS.Monitor/"]
RUN dotnet restore "DMS.Monitor/DMS.Monitor.csproj"
COPY . .
WORKDIR "/src/DMS.Monitor"
RUN dotnet build "DMS.Monitor.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "DMS.Monitor.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "DMS.Monitor.dll"]
When I try to build the image I get the following error:
#13 [build 5/8] RUN dotnet restore "DMS.Monitor/DMS.Monitor.csproj"
#13 sha256:f1029f328e1221cc75d3cbca3603f770dbb670711688ac30334449923e9ceb3f
#13 0.893 Determining projects to restore... #13 106.8 /src/DMS.Monitor/DMS.Monitor.csproj : error NU1102: Unable to find
package DevExtreme.AspNet.Core with version (>= 21.2.8) #13 106.8
/src/DMS.Monitor/DMS.Monitor.csproj : error NU1102: - Found 3
version(s) in nuget.org [ Nearest version: 20.1.3 ] #13 107.1
Failed to restore /src/DMS.Monitor/DMS.Monitor.csproj (in 1.77 min).
#13 ERROR: executor failed running [/bin/sh -c dotnet restore "DMS.Monitor/DMS.Monitor.csproj"]: exit code: 1
My project uses DevExtreme.AspNet.Core v.21.2.8 which is obviously not available in nuget.org. Is there a way I can edit the NuGet.Config file to tell Docker where it can find the correct version of the package?

You should be able to add your private feed using the command
dotnet nuget add source <package source>
Place it before the dotnet restore command in your Dockerfile.

Docker: "npm not found"

I'm trying to containerize an ASP.NET Core & Angular app but I'm having some trouble.
I'm getting these errors:
=> [internal] load build definition from Dockerfile 0.4s
=> => transferring dockerfile: 872B 0.1s
=> [internal] load .dockerignore 0.4s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for mcr.microsoft.com/dotnet/sdk:5.0 0.5s
=> [internal] load metadata for mcr.microsoft.com/dotnet/aspnet:5.0 0.0s
=> [base 1/7] FROM mcr.microsoft.com/dotnet/aspnet:5.0 0.0s
=> [build 1/7] FROM mcr.microsoft.com/dotnet/sdk:5.0#sha256:b69acf0a0734d77827d4e7ce22421256a64d490bb3ce988b21c4 0.0s
=> [internal] load build context 0.3s
=> => transferring context: 3.17kB 0.0s
=> CACHED [base 2/7] WORKDIR /app 0.0s
=> CACHED [base 3/7] RUN apt-get -y update 0.0s
=> CANCELED [base 4/7] RUN apt-get -y upgrade 5.6s
=> CACHED [build 2/7] WORKDIR /src 0.0s
=> CACHED [build 3/7] COPY [mediere.csproj, .] 0.0s
=> CACHED [build 4/7] RUN dotnet restore "./mediere.csproj" 0.0s
=> CACHED [build 5/7] COPY . . 0.0s
=> CACHED [build 6/7] WORKDIR /src/. 0.0s
=> CACHED [build 7/7] RUN dotnet build "mediere.csproj" -c Release -o /app/build 0.0s
=> ERROR [publish 1/1] RUN dotnet publish "mediere.csproj" -c Release -o /app/publish 4.7s
------
> [publish 1/1] RUN dotnet publish "mediere.csproj" -c Release -o /app/publish:
#21 1.567 Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET
#21 1.567 Copyright (C) Microsoft Corporation. All rights reserved.
#21 1.567
#21 2.460 Determining projects to restore...
#21 2.852 All projects are up-to-date for restore.
#21 3.883 mediere -> /src/bin/Release/net5.0/mediere.dll
#21 3.894 mediere -> /src/bin/Release/net5.0/mediere.Views.dll
#21 4.006 /bin/sh: 2: /tmp/tmp045a9cb1e4954d54b304a781ae210094.exec.cmd: npm: not found
#21 4.011 /src/mediere.csproj(38,5): error MSB3073: The command "npm install" exited with code 127.
------
executor failed running [/bin/sh -c dotnet publish "mediere.csproj" -c Release -o /app/publish]: exit code: 1
My dockerfile is this:
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 5000
#EXPOSE 5001
RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt install -y curl
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get install -y nodejs build-essential
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["mediere.csproj", "."]
RUN dotnet restore "./mediere.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "mediere.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "mediere.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "mediere.dll"]
I expected it to work, because I'm installing npm on line 12.
What I have observed is that the publish task and the apt-get install tasks work in the same time, and I think that publish goes before apt-get install and that's why it doesn't work. I might be wrong though.
How can I fix this error?
Thanks.

Each of the Dockerfile build stages starts FROM some other image. At the point you're using Node, that build stage is
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
...
FROM build AS publish
RUN dotnet publish "mediere.csproj" -c Release -o /app/publish
In this particular sequence of packages, starting from the .NET SDK image up through the point you run dotnet publish, but this never actually installs Node.
Where you do install Node it's in a base image stage. That's included into the final image, but not in any of the intermediate build steps.
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
RUN apt-get install -y nodejs
FROM base AS final
If you're building the Angular application into static files and then serving it from the .NET application, you only need Node during the build-and-publish phase, but not in the final application. I'd suggest installing it immediately after the FROM ... AS build line, and in particular before you COPY anything into the image. This will avoid reinstalling Node if you rebuild the image due to a code change.
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
# do not install Node here
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
# Do install Node
# The default version in the Debian repositories should be fine
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
build-essential \
nodejs
# Build your application as before
WORKDIR /src
...
FROM build AS publish
# Will include Node, because the `FROM ... AS build` stage installed it
FROM base AS final
# Will not include Node, because the `FROM ... AS base` stage did not
# install it

ASP.Net Core & Docker - Project file does not exist

I have this Dockerfile
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 5000
RUN apt-get -qq update && apt-get -qqy --no-install-recommends install wget gnupg \
git \
unzip
RUN curl -sL https://deb.nodesource.com/setup_6.x | bash -
RUN apt-get install -y nodejs
WORKDIR /app
#EXPOSE 5001
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["mediere.csproj", "."]
RUN dotnet restore "./mediere.csproj"
COPY . .
WORKDIR "/src/."
RUN apt-get -qq update && apt-get -qqy --no-install-recommends install wget gnupg \
git \
unzip
RUN curl -sL https://deb.nodesource.com/setup_6.x | bash -
RUN apt-get install -y nodejs
#RUN npm install zone.js#0.11.3 --save
WORKDIR /app
RUN dotnet build "mediere.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "mediere.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "mediere.dll"]
and running docker-compose up throws me this error:
=> ERROR [build 11/11] RUN dotnet build "mediere.csproj" -c Release -o /app/build 2.5s
------
> [build 11/11] RUN dotnet build "mediere.csproj" -c Release -o /app/build:
#23 2.241 Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET
#23 2.241 Copyright (C) Microsoft Corporation. All rights reserved.
#23 2.241
#23 2.245 MSBUILD : error MSB1009: Project file does not exist.
#23 2.245 Switch: mediere.csproj
------
executor failed running [/bin/sh -c dotnet build "mediere.csproj" -c Release -o /app/build]: exit code: 1
ERROR: Service 'web' failed to build : Build failed
and I don't see why.
I'm using a default ASP.Net core & Angular template from VS 2019 and I'm trying to dockerize it.
Here's the folder structure:
why am I getting this error?
Am I missing something obvious?

It's because you're setting the current working directory to /app right before building the project:
WORKDIR /app
The project doesn't exist in that directory. There should be no need to have that line at all because the working directory before this instruction gets executed is the correct one.

dotnet publish --self-contained option on Linux container

There are two ways to run a dot .net core application on a Linux based container.
Getting a base image which has .net core framework install (on top of Linux image) & then copy code using dotnet publish.
Use a Linux base image and then copy the dot net runtime & code using dotnet publish --self-contained option.
--self-contained [true|false]
Publishes the .NET runtime with your application so the runtime doesn't need to be installed on the target machine. Default is true if a runtime identifier is specified and the project is an executable project (not a library project).
Is there any advantage of using one option over other ?

The self-contained and trimmed app can be published in a container that only has runtime dependencies for dotnet without the need for the whole runtime. This will lead to much smaller images overall.
mcr.microsoft.com/dotnet/runtime:6.0-alpine -> 79.7MB
mcr.microsoft.com/dotnet/runtime-deps:6.0-alpine -> 10.1MB
For one of my apps:
dotnet publish -c Release -r linux-musl-x64 --self-contained false -> 30MB
dotnet publish -c Release -r linux-musl-x64 --self-contained true /p:PublishTrimmed=true -> 74.6MB
Final image:
Using runtime base: 120MB
Using runtime-deps base and trimming: 96.9MB
You can also cache the non-app binaries in a layer so when only app code changes only the layer with the app binaries will be pushed. Here is my setup for the whole context:
# Dockerfile-build
ARG BASE=mcr.microsoft.com/dotnet/sdk:6.0-alpine
FROM ${BASE} AS build
RUN apk add --no-cache rsync
WORKDIR /src
COPY *.sln .
COPY **/*.csproj ./
COPY **/**/*.csproj ./
COPY **/**/**/*.csproj ./
RUN dotnet sln list | grep ".csproj" | while read -r line; do mkdir -p $(dirname $line); mv $(basename $line) $(dirname $line); done;
WORKDIR /src
# The build has a memory leak if the proxy is not specified
# https://stackoverflow.com/questions/72885244/net-6-building-solution-at-docker-conteiner-taking-a-long-time-and-consuming
RUN export http_proxy=proxy:80
RUN export https_proxy=$http_proxy
COPY .cache* .cache
RUN --mount=type=cache,target=/root/.nuget/packages \
test -d .cache && rsync -a .cache/ /root/.nuget/packages/ && rm -rf .cache && echo "Cache applied"; \
dotnet restore -r linux-musl-x64
COPY . .
# We cannot use the -r flag for a global sln build https://github.com/dotnet/sdk/issues/14281#issuecomment-876510589
# The UnitTests project depends on all the other projects so this command builds the whole thing
RUN --mount=type=cache,target=/root/.nuget/packages \
dotnet build -c Release -r linux-musl-x64 -f net6.0 --no-restore Tests/IntegrationTests
FROM build as test
RUN --mount=type=cache,target=/root/.nuget/packages \
dotnet test -c Release -r linux-musl-x64 -f net6.0 --no-restore
FROM build as cache-prep
RUN --mount=type=cache,target=/root/.nuget/packages \
mkdir -p /packages && cp -R /root/.nuget/packages/* /packages
FROM scratch as cache
COPY --from=cache-prep /packages .
# To not trigger the stages above
FROM build
# Dockerfile-publish
ARG BASE=build
FROM ${BASE}
ONBUILD ARG DIR
ONBUILD ARG APP_NAME
ONBUILD ARG OUT_DIR=./out
# Set to 0 to disable trimming and layered publish
ONBUILD ARG TRIM=1
ONBUILD WORKDIR /src/${DIR}/${APP_NAME}
ONBUILD RUN --mount=type=cache,target=/root/.nuget/packages \
dotnet publish -c Release -r linux-musl-x64 --no-restore --no-dependencies \
$(test ${TRIM} -eq 1 && echo '--self-contained -p:PublishTrimmed=true') -o ${OUT_DIR}
# Move the app binaries to a different folder so we can try to cache the dependencies in a layer.
# That might not work for the trimmed build but if not much changed, this saves a lot of container space.
ONBUILD RUN test ${TRIM} -eq 1 \
&& mkdir -p ./app-bin \
&& mv ${OUT_DIR}/${APP_NAME}* ./app-bin \
&& mv ${OUT_DIR}/Kernel* ./app-bin \
|| echo 0
# Dockerfile-runtime
ARG BASE=publish
ARG RUNTIME=mcr.microsoft.com/dotnet/runtime-deps:6.0-alpine
# Name the layer so it can be used in a COPY command
FROM ${BASE} as publish
# Build runtime image
FROM ${RUNTIME}
# As per https://www.abhith.net/blog/docker-sql-error-on-aspnet-core-alpine/
RUN apk add --no-cache icu-libs tzdata \
&& cp /usr/share/zoneinfo/Europe/Prague /etc/localtime
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ENV TZ="Europe/Prague"
RUN adduser --disabled-password --home /app --gecos '' dotnetuser && chown -R dotnetuser /app
USER dotnetuser
ONBUILD ARG DIR
ONBUILD ARG APP_NAME
ONBUILD ENV APP_NAME=${APP_NAME}
ONBUILD ARG OUT_DIR=./out
ONBUILD WORKDIR /app
# TODO: use this once https://github.com/moby/buildkit/issues/816 is resolved
# For now just add the COPY commands to the project dockerfile
#ONBUILD COPY --from=publish /src/${DIR}/${APP_NAME}/out .
#ONBUILD COPY --from=publish /src/${DIR}/${APP_NAME}/app-bin .
ONBUILD ENTRYPOINT "./${APP_NAME}"
# project Dockerfile
ARG DIR=Service
ARG APP_NAME=XXXService
ARG BUILD=build
ARG PUBLISH=publish
ARG RUNTIME=runtime
FROM ${BUILD} as build
FROM ${PUBLISH} as publish
FROM ${RUNTIME}
# TODO: use ONBUILD in `publish` once https://github.com/moby/buildkit/issues/816 is resolved
COPY --from=publish /src/${DIR}/${APP_NAME}/out .
COPY --from=publish /src/${DIR}/${APP_NAME}/app-bin .
To build and publish you simply prepare the base images by
docker build -t build -f Dockerfile-build .
docker build -t publish -f Dockerfile-publish .
docker build -t runtime -f Dockerfile-runtime .
you test the app by
docker build -f Dockerfile-build --target test .
you can cache the nuget dependencies by exporting them from the image (in a CI environment for example)
docker buildx create --name buildx || true
docker build -f Dockerfile-build --builder buildx --target cache -o type=local,dest=./.cache .
and you build the final image by
docker build -t "$TAG" Services/XXXService
Hope this helps a little 🙂

Basically, you don't want self-contained as long as you don't need to. The package will stay as small as possible which is what you want. Only when you're not sure the targeting platform will be able to host your system because of missing runtime components, you can add these runtime components with the self-contained option.
You can install the dotnet runtime on several platforms including Linux. If you have control over doing so, I'd go for that option.