How to debug 203/EXEC when running dotnet appname.dll in redhat 7 systemd?

Question

Been trying to debug for 4 days now. I have an asp.net core 2 web app I'm trying to run in Redhat linux 7.1 systemd. The dll works fine when I run it manually, but fails when trying to run through systemd.

Output from systemctl

web.service - WebServiceLayer Loaded: loaded (/etc/systemd/user/web.service; enabled; vendor preset: >disabled)

Active: activating (auto-restart) (Result: exit-code) since Sun 2019-04-28 >12:49:01 CDT; 2s ago

Process: 13588 ExecStart=/microsoft/dotnetcore/dotnet /local/lfs1/ServiceLayer/WebServiceLayer/WebServiceLayer.dll (code=exited, status=203/EXEC)

Main PID: 13588 (code=exited, status=203/EXEC)

Apr 28 12:49:01 mymachine.net systemd[1]: web.service: main process exited, code=exited, status=203/EXEC

Apr 28 12:49:01 mymachine.net systemd[1]: Unit web.service entered failed state.

Apr 28 12:49:01 mymachine.net systemd[1]: web.service failed.

Here's the basic service file

mymachine home/username $ cat /etc/systemd/user/web.service
[Unit]
Description=WebServiceLayer
After=syslog.target

[Service]

User=serviceUser

Group=serviceUser
PIDFile=/tmp/$i.pid
WorkingDirectory=%h
Environment="PATH=/microsoft/dotnetcore/"
ExecStart="/microsoft/dotnetcore/dotnet /local/lfs1/ServiceLayer/WebServiceLayer/WebServiceLayer.dll"
Restart=always
RestartSec=3
StartLimitBurst=10
StartLimitIntervalSec=0
KillSignal=SIGQUIT
StandardError=syslog
NotifyAccess=all

[Install]
WantedBy=multi-user.target

For reference, when I start the dll normally I get no errors:

[serviceUser@mymachine]: /microsoft/dotnetcore/dotnet /local/lfs1/ServiceLayer/WebServiceLayer/WebServiceLayer.dll Hosting environment: Production Content root path: /home/serviceUser Now listening on: "http://0.0.0.0:7777" Application started. Press Ctrl+C to shut down.

I have looked at Fixing a systemd service 203/EXEC failure (no such file or directory) but that didn't help my issue. Any help would be greatly appreciated.

Code from Program.cs in .dll file:

public static void Main(string[] args)
        {
            // NLog: setup the logger first to catch all errors
            var logger = NLog.Web.NLogBuilder.ConfigureNLog("nlog.config").GetCurrentClassLogger();
            try
            {
                logger.Debug("init main");
                CreateWebHostBuilder(args).Build().Run();
            }
            catch (Exception ex)
            {
                //NLog: catch setup errors
                logger.Error(ex, "Stopped program because of exception");
                throw;
            }
            finally
            {
                // Ensure to flush and stop internal timers/threads before application-exit (Avoid segmentation fault on Linux)
                NLog.LogManager.Shutdown();
            }
        }

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseUrls(urls: "http://mymachine.net:7777") 
                .UseStartup<Startup>()
                .ConfigureLogging(logging =>
                {
                    logging.ClearProviders();
                    logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
                })
                .UseNLog();  // NLog: setup NLog for Dependency injection
    }

Maarten Kieft Maarten Kieft · Accepted Answer · 2019-04-28T18:30:15

Well from the output you get it's hard to know what is going on. I would advise you yo configure some extra logging.

First you could configure logging to a text file to get more details. Some more info: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-2.2

After that you could place a big try / catch around the code in the main method and log the exception.

The only quick check which comes in my mind, does the serviceUser has enough rights to access the dll or config files or any related stuff?

How to debug 203/EXEC when running dotnet appname.dll in redhat 7 systemd?

1 Answers