Install infinite loop

PJPorch

@OutagesIO_Support This is installed on an Ubuntu server, so I am using the linux script. The reason I posted here is I am seeing the exact same issue where it keeps trying to download the otm_linux binary and can never successfully do it. I can repost to the Linux section if required. I was just thinking the symptoms are identical, so the solution might be the same.

I have also tried to disable the service, adjust the script as below, and then run in manually (screenshot). I get the same result where it loops trying to download the file.

Script when service is enabled:

## exec $start/otm_linux &
#$start/otm_linux &

Script when service is disabled and the script is run manually:

# exec $start/otm_linux &
$start/otm_linux &

Here is my service file. Located at: /etc/systemd/system/otm.service

[Unit]
Description=Startup of the OutagesIO monitoring agent

[Install]
WantedBy=multi-user.target

After=network.target

[Service]
Type=simple
ExecStartPre=/home/josh/agent/starter_linux_ocp.sh
ExecStart=/home/josh/agent/otm_linux
WorkingDirectory=/home/josh/agent
Restart=always
RestartSec=5
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=%n

Results when running the script directly.

OutagesIO_Support

Yes, it doesn't seem to be finding the binary. Can you try this manually.
Go into the same directory and run both;

curl http://downloads.outagesio.com/otm/otm_1.67.2104_linux -o otm-linux

See if that downloaded the binary. Delete it and try https.

curl https://downloads.outagesio.com/otm/otm_1.67.2104_linux -o otm-linux

If it's there, make it executable then run the version.

$ chmod +x otm-linux
$ ./otm-linux -v

Show me the output please.

PJPorch

@OutagesIO_Support Both the HTTP & HTTPS links successfully download the file. I am able to get the version following your commands as well.

PJPorch

@PJPorch In looking at the script I notice the URL you gave to directly download the file is from outagesio.com, but the script goes to foxymon.com. Could there be an issue with this foxymon.com URL?

BINARY_LOCATION=$(curl -m 20 -s -u $USERNAME:$PASSWORD --connect-timeout 10 -X POST https://www.foxymon.com/receiver/receiver2.php -F function=receive_binary_location)

OutagesIO_Support

Yes, it's correct. Foxymon.com is a domain we started with years ago and never got around to changing it since it would require a ton of programming so we just use it in the background.

The script checks a function which tells it what the download URL is based on the credentials you enter into /etc/agentid.

I'm not sure if you shared this before but can you share the agent ID for this install as well please.

I see three which seem to be yours. 128600 which is active, 128588 which is also active and 128593 which is deleted.

PJPorch

@OutagesIO_Support
Agent 128588 is working without any issues.
This post is regarding Agent 128600.
The 3rd must have been one I created during testing then deleted.

OutagesIO_Support

@PJPorch said in Install infintie loop:

This post is regarding Agent 128600.

This agent is currently running, it's in Active status.

PJPorch

@OutagesIO_Support

Nothing is being populated in the ping field or recent events.
The public IP is not updated to the correct address.
The service on the Ubuntu machine shows it is running, but I cannot tell if it was able to download the agent or not. Manually downloading might have helped this get started.

OutagesIO_Support

Wait now, one thing at a time. We need to be in sync.
You said that you were not able to run the agent at all, that it kept downloading but that's all.

The script shows you can use this to see if the agent is running.

ps cax | grep otm_linux

Can you check so we can see if there are one or more instances running.

Since we see it running, kill it so we have a starting point.

The first thing is to download the agent on it's own, without using the script, make it executable then start it then run the above command to confirm it is running and only one is.

If you see it running, then kill the process and we'll do this next.

Next, try this.

curl -m 20 -s -u xxxx:xxxxxxxxxxxxxxxxxx --connect-timeout 10 -X POST https://www.foxymon.com/receiver/receiver2.php -F function=receive_binary_location

The above is from the script. You replace the xxxx:xxxxxxxxxxxxxxxxxx with the credentials you have /etc/agent. Note the semicolon.

What this does is to query your account for the new agent you've created so it can give you the latest version of the agent for that particular OS. Show me the output of this please.

We'll deal with the rest one thing at a time to prevent confusing each other.

PJPorch

@OutagesIO_Support
Sorry...I guess I was jumping ahead when you said you showed the agent as active.

I have stopped the service from running.

sudo service otm status

If I search for the running process with your script I get nothing.

ps cax | grep otm_linux

If I search for the name of the script I do get some results, so I manually have killed all processes running the script,

ps -aux | grep 'starter_linux_ocp'

When I run the curl command with my credentials I get a response of
"https://downloads.outagesio.com/otm/otm_1.67.2104_linux" which looks like it exactly matches the URL I was able to manually download from, so that part of the script seems to be working correctly.

OutagesIO_Support

I think the first thing would be to kill everything that's been left over from your testing so you are starting from scratch. make sure the service is stopped, all 'starter' scripts processes are killed.

Then check your dashboard until the agent is Inactive. Once you know there is no agent or anything else related running, start your agent manually, not using systemd and check if the agent becomes active.

Once sure there is nothing else running and the agent goes active, then kill that process and start the agent using your systemd start command. I don't know for sure but it looks like that part was working, only there were other instances running too which could have caused conflicts maybe. Let's get to the point of the agent running and we'll look into the other stuff after.

PJPorch

@OutagesIO_Support

Disabled the service
Killed all running processes
Agent shows as "Inactive"
Edited the script to run manually.

# exec $start/otm_linux &
$start/otm_linux &

Run the script manually and I'm right back at the beginning where it won't download the file. There are no apparent errors with credentials, permission to download the file, or anything indicating why it can't download the file.

OutagesIO_Support

You seem to be doing all the right things. It's a little perplexing to say the least. It must be something small and so obvious that it's not obvious.

We will dig deeper into this tomorrow because it really isn't making sense and must be something on our end.

OutagesIO_Support

Ok, some good news for you. The problem should be resolved now.
Just create a new agent then download the updated script, version 1.0.5. You can let that new agent get deleted automatically by the system in 24hrs. Use that script and as before, make sure to upgrade the path's of course.

It should work now.

Also made a small change in the systemd if you're using it. Still have to solve why it's echo'ing out the text from the script but it works.

[Unit]
Description=Startup of the OutagesIO Internet monitor agent
After=network.target

[Install]
WantedBy=multi-user.target

[Service]
Type=simple
ExecStartPre=/otm/starter.sh
ExecStart=/otm/otm_binary
WorkingDirectory=/otm
Restart=always
RestartSec=5
StandardError=syslog
SyslogIdentifier=%n

Give it a try, let us know how it goes.

PJPorch

@EchoMin
Whatever changes you made to the script seems to have fixed it. I deleted everything, reinstalled with the new scripts, and everything seems to be downloading and running properly.

Thanks for all the help.
Josh

OutagesIO_Support

Nice to hear.

Which one is your most important agent ID? One that you'll keep going without deleting? We'll keep an eye on that for a bit too.

PJPorch

@OutagesIO_Support The new agent is 128834. This was the site with all the issues that seems to be running well now.

Agent 128600 can be deleted.

OutagesIO_Support

We gave you a free upgrade on that agent as a thanks for sharing this problem with us.

Most people just leave before telling us there's a problem we could fix.

Thanks.

PJPorch

@OutagesIO_Support Thank you. I appreciate the time it took to troubleshoot this one with me.

Information and Support

Install infinite loop