Agents is running but not reporting

[lp4y]

The agent 128994 which is under the latest release has already stopped 9 times this morning, see below the logs.

(ON means the monitor is triggered because the agent is not running, OFF means the monitor is not triggered because the agent is running).

2022/06/28 12:33:18 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:33:14 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:31:37 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:31:33 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:30:01 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:29:57 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:26:52 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:26:48 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:25:14 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:25:10 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:08:28 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:08:24 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:06:21 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:06:17 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:05:09 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:05:05 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/28 12:01:28 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/28 12:01:22 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

[lp4y]

Just found some errors in the Windows Event viewer (all the errors are from OtmWinClient.exe) :

[OutagesIO_Support]

That's great, thanks. Any more logs or information you might see that can be shared?

[OutagesIO_Support]

Why don't you change the apps to admin in the properties box (compatibility tab \ run as administrator) and see if that helps.

[OutagesIO_Support]

Only the top-most entry is shown. So it would be interesting to see if the others have the same “fault offset”

[OutagesIO_Support]

Can you let me know which version specifically please.

Standard or standard with desktop or dc or dc with desktop?

[lp4y]

This post is deleted!

[lp4y]

Standard with desktop experience

[lp4y]

@OutagesIO_Support said in Agents is running but not reporting:

Why don't you change the apps to admin in the properties box (compatibility tab \ run as administrator) and see if that helps.

I prefer not to try that unless you have a good reason to think it can solve the issue.

[lp4y]

@OutagesIO_Support said in Agents is running but not reporting:

That's great, thanks. Any more logs or information you might see that can be shared?

Not from the event viewer I guess. Is there any application logs somewhere ?

[OutagesIO_Support]

To be transparent, we're not trying random things, we're actually working on this in the lab.

So far, we are able to find some small things we can improve on but have not been able to replicate this. On our 2016 test, it's working non stop, the service does not get turned off. Therefore, we need you to try things on your server that we don't have access to in order to get some feedback otherwise, it's hard to help.

Right now, we could use more logs from your server so the agent developer can have some leads.

We really want to see this working for you. We think it's important because we aren't really supporting servers and maybe we should.

Most Enterprise customers use hardware agents because they don't have to deal with any of this. They install the agent, power it up, it's done. The agent self updates when and as needed and we can even troubleshoot them from remote.

We have to keep in mind that every server out there is different from others. Even if your are all the same to each other, they are not the same as other win2016 servers. Combinations of settings, software installed, countless things make every server unique.

Anyhow, we appreciate your patience and if you are willing to continue, we are working on it in the background.

[lp4y]

I have enabled to run OtmWinClient.exe as Administrator on agent 128994 :

I'll let you know if I see the agent stopping and being restarted.

[lp4y]

I didn't solve the problem, the agent stopped 5 times today :

2022/06/30 03:00:49 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/30 03:00:47 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/30 02:59:25 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/30 02:59:23 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/30 02:55:29 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/30 02:55:27 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/30 02:50:32 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/30 02:50:30 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

2022/06/30 02:46:35 PM	OFF	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is running

2022/06/30 02:46:29 PM	ON	

Process Monitor : Condition: OtmWinClient.exe is not running - Status is not running

[OutagesIO_Support]

I'm not a Windows person so not sure what to look for specifically but I know that in the events manager, the system will keep a lot of lots and usually shows why a service would be shut down. It doesn't seem to be crashing, it seems to be being shut down by the server. There is perhaps a policy or something kicking in.

[lp4y]

I think the app is crashing, see from the event viewer :

[OutagesIO_Support]

Just so you know, we were able to replicate this on a Windows 7 machine so are digging into it.

[OutagesIO_Support]

Hi again,

Just wanted you to know that this is something we are still working on. Because it's so random, it's been extremely time consuming trying to nail it down.

I'll post an update once we solve the problem and at that point, it should work on your servers.

[lp4y]

Ok, thanks for the update.

[OutagesIO_Support]

As yet another update. We thought we had this fixed so applied the new code and let it run for almost a week and it died again. Still trying to understand what is causing this and have not found the lead.

This is partly why agent updates and releases can sometimes take quite a while as we have to run them for weeks to make sure we can find and squash bugs. This one is quite elusive but once we solve it, it'll be good to go.

[lp4y]

Thanks for the heads up.