Track Internet disconnections, provider outages with historical data, and automated speed testing.
For Windows, Linux, ARM64, ARMa7. Learn more by visiting www.outagesio.com
Notice: If you created an account on app.outagesio.com, simply use the same credentials to log in here.
Agents is running but not reporting
-
Happy that you stuck with us to this point.
Can you let us know how things go once you feel the agents are working as expected. As SBK mentioned, feedback is highly important to us and helps to improve where ever we can.
Hello.
I have updated all my agents to v1.76 but today I noticed some agents are still not working. E.g. agent 129005 is reported disconnected from the portal :
but the agent is actually running on the server :
and ping from this agent to your server is also working :
I can't see any error in Windows logs.
Restarting the service brought the connection back with your portal.
I have this problem on couple of servers running Windows Server 2016 and also on a PC Running Windows 10 (21H2).
Any idea ?
-
I'll pass this on to our windows dev. It's sad to see this since we spent months trying to find the problem so this is a bit confusing.
Since you don't see anything in the logs, then we need to look at our end to see when the agent(s) stops communicating to see if there is a pattern. Can you provide all the agent IDs involved so we can do that.
The agent is supposed to auto restart every 24hrs so it's possible something is preventing it from doing that and maybe it ends up crashing in memory.
Maybe the solution is to auto restart the service daily. Is that something you could try?
We will look at our end to see if it's going offline around the time it's supposed to restart.
Question
We see 30 agents installed but most of those are always disconnected status.You mentioned these are all Windows 2016 servers so are some of those turned off at night or something or are they supposed to be running 24/7?
I ask because looking into this, we cannot know which servers are supposed to be (or are) left on 24/7.
-
I'll pass this on to our windows dev. It's sad to see this since we spent months trying to find the problem so this is a bit confusing.
Since you don't see anything in the logs, then we need to look at our end to see when the agent(s) stops communicating to see if there is a pattern. Can you provide all the agent IDs involved so we can do that.
The agent is supposed to auto restart every 24hrs so it's possible something is preventing it from doing that and maybe it ends up crashing in memory.
Maybe the solution is to auto restart the service daily. Is that something you could try?
We will look at our end to see if it's going offline around the time it's supposed to restart.
Question
We see 30 agents installed but most of those are always disconnected status.You mentioned these are all Windows 2016 servers so are some of those turned off at night or something or are they supposed to be running 24/7?
I ask because looking into this, we cannot know which servers are supposed to be (or are) left on 24/7.
I don't know which agents are involved and I guess it's happening on all of them.
The servers are only on during daytime as they are used for computer labs. They sometimes stay powered on during the night if the students forget to turn them off when they leave the training center.
There is no point of configuring a daily auto-restart as most of the servers are shutdown at night. And some agents which have been up for less than 24hours are reported disconnected.
E.g..at the time of writing this message (11:55am IST time), agent 129006 has been up for 21 hours and is reported disconnected.
-
I asked the Windows dev and he says it's not possible that the agent is not logging what it's doing. Perhaps you're not looking in the right place?
Also, you mention that most of the machines are offline but you said you guess it's happening to all of them.
The problem with that is that it makes it much harder for us to know where to search for problems :).
We absolutely want to see this working for you and right now, you've told us the new version may not be working after we've spent many months testing so this comment makes us quite nervous.
Is there any way you can confirm how many might be having problems? There should be something in the logs if you could take a deeper look. I think it would be in the application log.
-
I asked the Windows dev and he says it's not possible that the agent is not logging what it's doing. Perhaps you're not looking in the right place?
Also, you mention that most of the machines are offline but you said you guess it's happening to all of them.
The problem with that is that it makes it much harder for us to know where to search for problems :).
We absolutely want to see this working for you and right now, you've told us the new version may not be working after we've spent many months testing so this comment makes us quite nervous.
Is there any way you can confirm how many might be having problems? There should be something in the logs if you could take a deeper look. I think it would be in the application log.
I found out that the agents were not working by comparing how many agents are running on your portal and my RMM tool. As it is a manual comparison, I can't determine which agents have this problem exactly, but it seems to be random.
You need to point me where exactly to log for the logs. I've looked in the Windows application logs and couldn't see anything.
-
Hi,
Yes, it's the application log. If you're not seeing anything being logged about the agent, maybe that's a lead to something else. We have never seen it not log.
As for random, not trying to get out of this as we'd love to solve it but it's not enough information to know what's going on.
Random, no logs, really not sure how to help.
-
Hi,
Yes, it's the application log. If you're not seeing anything being logged about the agent, maybe that's a lead to something else. We have never seen it not log.
As for random, not trying to get out of this as we'd love to solve it but it's not enough information to know what's going on.
Random, no logs, really not sure how to help.
Yes I understand it's difficult for you to troubleshoot.
I can already confirm to you that these agents are running into this problem :
129390
129006
129247
129417
129793I'm willing to do more troubleshooting but I have no clue what to do.
All I can tell is the agent is running, the communication to your servers is working but it's reported disconnected on the portal.
-
Yes I understand it's difficult for you to troubleshoot.
I can already confirm to you that these agents are running into this problem :
129390
129006
129247
129417
129793I'm willing to do more troubleshooting but I have no clue what to do.
All I can tell is the agent is running, the communication to your servers is working but it's reported disconnected on the portal.
We'll try to see if we can spot anything useful using those agents next week. In the meantime, if you can keep an eye on the app logs and see if you can find us any logging, that's what we really need.
There is one more thing. we could give you a special version of the agent that you could install on some of the machines you've confirmed are doing this.
The agent logs to a file so any problems would be logged even if it's not in the Windows app logs for some reason.
-
We'll try to see if we can spot anything useful using those agents next week. In the meantime, if you can keep an eye on the app logs and see if you can find us any logging, that's what we really need.
There is one more thing. we could give you a special version of the agent that you could install on some of the machines you've confirmed are doing this.
The agent logs to a file so any problems would be logged even if it's not in the Windows app logs for some reason.
-
Yes I understand it's difficult for you to troubleshoot.
I can already confirm to you that these agents are running into this problem :
129390
129006
129247
129417
129793I'm willing to do more troubleshooting but I have no clue what to do.
All I can tell is the agent is running, the communication to your servers is working but it's reported disconnected on the portal.
-
Yes I think giving me a agent with verbose logs enabled is a good idea, I can deploy that on a few servers.
-
We have a custom version ready for you. Just need you to respond to the email we've sent :).
-
Hi, have you had the chance to install the new agent?
-
Just checking in. Would love to know how it's going :).
-
Just checking in. Would love to know how it's going :).
-
Well that's good news. I assume you installed it on a server you were seeing those service problems with.
The v1.77 has some additional updates that we've been working on.
Here's a post about it. About v1.77.2207
The main things are;
-Improved thread management was a core requirement in order to prevent the potential loss of occasional data being sent to dashboard.
-Improved Windows service function to prevent the service from potentially stopping on some Win machines.
-For those using Extended reports, automated speed testing now shows downloads, uploads and latency.
Also, we have lowered our pricing. A subscription to Extended is only $4.00/month for the holidays.
-
Today I noticed agent 129006 was online in my tools but it was reporting disconnected from outages.io portal.
I stopped the service, copied the log files.
After restarting the service, the agent is showing online again on your portal.I looked at the logs and I can access the IP address the agent is trying to connect to from the host so it is not a connectivity issue :
Let me know if you find something interesting.
-
Today I noticed agent 129006 was online in my tools but it was reporting disconnected from outages.io portal.
I stopped the service, copied the log files.
After restarting the service, the agent is showing online again on your portal.I looked at the logs and I can access the IP address the agent is trying to connect to from the host so it is not a connectivity issue :
Let me know if you find something interesting.
Hi again,
Thanks for sharing this.
Got the log, (removed the shared url).
Looking at the log, the agent never crashed or stopped so what ever happened doesn't seem to be the agent itself that is stopping but something is preventing it from communicating.
Is it possible that agents are being blocked by something at the network edge or with the provider for some reason or another.
-
Hi again,
Thanks for sharing this.
Got the log, (removed the shared url).
Looking at the log, the agent never crashed or stopped so what ever happened doesn't seem to be the agent itself that is stopping but something is preventing it from communicating.
Is it possible that agents are being blocked by something at the network edge or with the provider for some reason or another.
