Change font size
It is currently Thu Nov 21, 2019 7:01 pm

Forum rules


{L_IMAGE}
. please keep questions related to an OpenSimulator and OSgrid nature.



Post a new topicPost a reply Page 1 of 1   [ 5 posts ]
Author Message
 Post subject: Closing down a single region takes over an hour.
PostPosted: Thu Jul 25, 2019 5:11 am 
Furious Typer

Joined: Tue Oct 28, 2008 4:53 pm
Posts: 172
Before anyone asks, yes I have the latest OSgrid version of opensim, and mono V 6.0.0.313, with headless Ubuntu Server 18.0.4 LTS and mysql Ver 14.14 Distrib 5.7.27, for Linux (x86_64).

The server is a Dell Poweredge T710 with 2x Intel Xeon 6 core CPU's, 192GB RAM, 8x 1TB SAS HD's configured to raid 5.

OK, now, here's the biggest problem with closing some regions, one of the regions I have a single 256x256 region with a couple NPC's that randomly move from point to point, and the region has 120 prims and 35 active scripts, now when I close down the region, perhaps to update opensim, using quit command, it gets to 'Persisting changed objects....' then hangs there, thinking it froze I used CTRL-A-D (I'm using screen under linux to run each simulator as a separate instance), which tells me the system hasn't froze, so I used 'screen -r <region name>' to go back to the console, and I waited, and waited, and waited, for what seemed a very long time then it closed on it's own, so this time I got a stopwatch and timed it,,,, the shutdown from when I issued 'quit' to when it actually exited from screen and went to command prompt took exactly 64 minutes, and 43 seconds, while it was stuck on 'Persisting changed objects.....'

Sooo, from then on I thought 'OK, this must be a bug' so I don't use 'quit' anymore I just use 'killall screen' to shut them down, after checking first using 'show users' to make sure nobody is on the region(s), and using 'killall screen' shuts them down a lot quicker if I wanted them all shutdown, but if I wanted just one just down, I would use 'kill <PID>' after first using 'screen -list' to see the PID of the region I want to shutdown.

There has to be a quicker way to shut down a region without killing it, or waiting for over an hour, 1-2 minutes I can tolerate, but over an hour? yeah right.


Top
 Profile  
 
 Post subject: Re: Closing down a single region takes over an hour.
PostPosted: Thu Jul 25, 2019 9:07 am 

Joined: Thu Aug 16, 2018 6:41 am
Posts: 29
Hi Marlon,
I use shutdown myself rather than quit, not sure what the difference is.

Trying doing a backup while you are looking at the console you may see some messages. If you then shutdown, there will be nothing to persist to the database so it should be quicker and cleaner.

Regards
Baron


Top
 Profile  
 
 Post subject: Re: Closing down a single region takes over an hour.
PostPosted: Thu Jul 25, 2019 12:47 pm 
Furious Typer

Joined: Tue Oct 28, 2008 4:53 pm
Posts: 172
Baron Brown {L_WROTE}:
Hi Marlon,
I use shutdown myself rather than quit, not sure what the difference is.

Trying doing a backup while you are looking at the console you may see some messages. If you then shutdown, there will be nothing to persist to the database so it should be quicker and cleaner.

Regards
Baron


There is no difference between 'quit' and 'shutdown', either one does the same thing.

I just tried using 'backup' and there was no messages, but it still hung on 'persisting changed objects' when I used either 'shutdown' or 'quit', it seems when the thread that compares a region from before a shutdown to when it's shutdown is running, it sees a change from when an NPC has moved from one position to another and decides there's been a change and triggers 'persisting changed objects', it seems to only happen when an object or a group of objects are moving, regardless of if an NPC or a scripted object, if you drove a car from one spot to another then shutdown, it too triggers the 'persisting changed objects', I tested this with creating a new 256x256 single region on its own instance, using its own database (this is so that I can start a fresh if I need to without affecting any other region), then I used an oar with nothing but stationary objects and no scripts, a quit/shutdown was almost instant, now I deleted the DB and recreated a new one, then used an oar with 500 objects and 66 active scripts, some of those objects are scripted physical objects which move, then a shutdown/quit resulted in 'persisting changed objects....' and hung, this time for 38 minutes and 14 seconds.

Maybe I should use 'kill' to instantly kill a region and not wait from 1/2 an hour to over 1 hour. Even using 'kill' the objects are still there when I fire up the region again, so what is this 'persisting changed objects", anyway?

The problem when using 'kill' is that the region is not de-registered on the OSG grid database, and ends up as a phantom region on the OSG grid map, and the only way to remove it, is to login to an OSG account on OSG website, then using one's account to 'purge' the region(s) from the grid database.


Top
 Profile  
 
 Post subject: Re: Closing down a single region takes over an hour.
PostPosted: Thu Jul 25, 2019 8:13 pm 

Joined: Thu Aug 16, 2018 6:41 am
Posts: 29
Hi Marlon,
Would using scripts stop or scripts suspend via screen work? I don't know if there is a message channel that could be fired from the simulator, but if that could issue a kill command for the NPCs?

On restart, just resume the scripts.

Just throwing some ideas

Baron


Top
 Profile  
 
 Post subject: Re: Closing down a single region takes over an hour.
PostPosted: Sat Jul 27, 2019 9:49 am 
Furious Typer

Joined: Tue Oct 28, 2008 4:53 pm
Posts: 172
I just did some more testing, and it's not just NPC's, but also any physical scripted objects that are moving, such as the Satyr farm animals, or aquatic animals such as dolphins, whales, fish etc etc if it's physical and moves either constantly or randomly (non-physical objects that use a rotation script are not affected), then for some reason when the region is shut down, both 'backup' and 'quit' trigger the 'persisting changed objects....' and can take over an hour to shutdown, so, being there is no viable solution, I'll just have to use 'kill' to instantly stop any region or regions, then use my OSG account to 'purge' the regions from the grid database, which effectively 'de-registers' the region(s) just as if using the 'quit' or 'shutdown' command.

Being that I use 'screen' to run multiple sim instances, each with their own region DB, but a centralized estate DB for all regions, and if I need to either update the OS, mono or opensim, then I'll have to use 'killall screen' to instantly stop all regions, after first running a bash script that opens each screen so that I can use 'show users' to make sure nobody is on the regions, then I use another bash script to start each region again after the update(s). That's the only viable solution that works without having to wait hours for some regions to shutdown.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post a new topicPost a reply Page 1 of 1   [ 5 posts ]


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  


Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
610nm Style by Daniel St. Jules of Gamexe.net