GSP/Panel

Fork 0

Frank Harris 3829a4a83d worshop work

2026-06-08 16:09:54 -05:00

25 KiB

Raw Blame History

GSP Scheduler Actions Design

Workspace reference: GSP-WORKSPACE.md

Scope

This is an investigation and design report only. It does not implement code.

The goal is to redesign GSP's Scheduler / CRON feature into a safer, more useful automation system for game hosting customers and administrators.

Repository layout reviewed:

Agent-Windows
Agent_Linux (the Linux agent directory currently uses an underscore in this repository)
Panel
Website

1. Current Scheduler Module Findings

Files inspected

Panel Scheduler module:

Panel/modules/cron/module.php
Panel/modules/cron/navigation.xml
Panel/modules/cron/cron.php
Panel/modules/cron/user_cron.php
Panel/modules/cron/shared_cron_functions.php
Panel/modules/cron/events.php
Panel/modules/cron/thetime.php

Panel remote/API integration:

Panel/includes/lib_remote.php
Panel/includes/api_functions.php
Panel/modules/gamemanager/start_server.php
Panel/modules/gamemanager/stop_server.php
Panel/modules/gamemanager/restart_server.php
Panel/modules/gamemanager/update_actions.php
Panel/modules/gamemanager/rcon.php
Panel/modules/addonsmanager/server_content_actions.php

Agent scheduler implementation:

Agent_Linux/ogp_agent.pl
Agent-Windows/ogp_agent.pl

Current database tables used

The current Scheduler module does not appear to own database tables. Module metadata has:

Panel/modules/cron/module.php
$db_version = 0

Scheduled jobs are stored on each agent in a flat file:

Linux: AGENT_RUN_DIR/Schedule/scheduler.tasks
Linux: AGENT_RUN_DIR/Schedule/scheduler.pid
Linux: AGENT_RUN_DIR/Schedule/scheduler.log
Windows/Cygwin: AGENT_RUN_DIR/scheduler.tasks
Windows/Cygwin: AGENT_RUN_DIR/scheduler.pid
Windows/Cygwin: AGENT_RUN_DIR/scheduler.log

This means the agent is currently the storage location for task definitions, and the Panel reconstructs task lists by asking each agent for its task file.

Current actions

Current customer-visible scheduled actions from get_action_selector():

restart
stop
start
steam_auto_update when the game XML installer is steamcmd

Additional server content actions are appended when addonsmanager is installed:

server_content_check_updates
server_content_check_workshop_updates
server_content_install_updates_if_stopped
server_content_install_updates_next_restart
server_content_install_updates_now
server_content_install_updates_and_restart
server_content_notify_updates_only
server_content_update_all
server_content_validate_files
server_content_backup_before_update

Admin-only raw command path:

Panel/modules/cron/cron.php exposes a second form where an admin selects a remote server and enters a raw shell command.

How tasks are created

Customer task creation path:

User opens Panel/modules/cron/user_cron.php.
User selects a game server and action.
Panel validates the five CRON fields using checkCronInput().
Panel calls build_cron_scheduler_command().
The command is built as a wget callback to ogp_api.php.
Panel sends the whole CRON line to the agent through scheduler_add_task().
Agent appends the task line to scheduler.tasks.
Agent restarts its scheduler process.

Admin task creation path:

Admin opens Panel/modules/cron/cron.php.
Admin can use the same server/action selector.
Admin can also enter a raw command for a remote server.
Panel writes the raw command into the agent task file.

Current scheduled API callback examples:

wget -qO- "<panel>/ogp_api.php?gamemanager/stop&token=<token>&ip=<ip>&port=<port>&mod_key=<mod_key>" --no-check-certificate > /dev/null 2>&1

wget -qO- "<panel>/ogp_api.php?server_content/run_scheduled_action&token=<token>&home_id=<home_id>&action=<action>&options=<json>" --no-check-certificate > /dev/null 2>&1

How tasks execute

Both agents use Perl Schedule::Cron.

Agent startup:

Stops prior scheduler process using scheduler_stop().
Creates a Schedule::Cron object.
Adds a read/reload task that runs every second:
- * * * * * *
Starts scheduler detached with scheduler.pid.

Agent task reload:

scheduler_read_tasks() opens scheduler.tasks.
It clears the in-memory timetable.
It splits each line into five CRON fields plus command args.
If args start with %ACTION, it uses scheduler_server_action().
Otherwise it adds a generic shell command task.

Current Panel-generated jobs are generic shell commands, not %ACTION jobs. They execute through:

scheduler_dispatcher()
backtick execution of the scheduled command
append to scheduler.log

The older %ACTION=start|stop|restart direct-agent scheduler path still exists but does not appear to be the primary current Panel path.

How task results are logged

Agent logging:

scheduler_log_events() appends plain text to scheduler.log.
Generic commands log:
- the command text
- any response text

Panel viewing:

Panel/modules/cron/events.php reads scheduler.log from the selected remote server.
It refreshes the log area periodically.

Limitations:

No structured per-task run records.
No status model such as pending/running/success/failed/skipped.
No reliable per-run output attached to a task ID.
No last run / next run / duration / exit code storage in the Panel DB.
wget callbacks redirect output to /dev/null, so useful API responses are discarded.

Current limitations and bugs

No Scheduler-owned database tables.
Tasks are stored per agent, so offline agents make task state invisible or stale.
Tasks contain API tokens in plain text inside agent task files.
Generic command scheduler can run arbitrary shell commands.
Admin raw command scheduling is powerful and should remain admin-only or be removed from the normal Scheduler UI.
Current customer tasks call the Panel through wget, so task execution depends on the agent reaching the Panel HTTP URL.
--no-check-certificate weakens TLS verification.
Task output is discarded for Panel API callbacks.
No retry policy.
No overlap prevention.
No conflict prevention, such as update and restart at the same time.
No job lock per game server.
No missed-run handling after agent downtime.
No clear timezone UX.
Admin and customer scheduling models are mixed in the same module.
Server content scheduled actions include duplicates and placeholders.
Some action names are not customer-friendly.
There is no typed argument system for warnings, backup paths, retention, RCON command allowlists, or wipe options.
There is no first-class notification support.
Linux and Windows store scheduler files in different relative locations.

2. Current Action Review

Action	Keep/Remove/Admin Only	Why	Security Risk	Agent Support	Notes
`restart`	Keep	Core hosting feature.	Low if implemented through safe action.	Existing Panel API and agent restart support.	Should support warnings, save-world, lock, timeout, and result logging.
`stop`	Keep	Useful for scheduled shutdown windows and cost/resource control.	Low.	Existing Panel API and agent stop support.	Should verify stopped state through agent status.
`start`	Keep	Useful after maintenance windows.	Low.	Existing Panel API and agent start support.	Should show STARTING/ONLINE result, not only command fired.
`steam_auto_update`	Keep, rename	Useful but name is technical.	Medium due Steam credentials/update side effects.	Existing `steam_cmd` update path.	Rename to `update_server_files`; require game XML installer support.
`server_content_check_updates`	Keep internally, remove from customer dropdown for now	Useful as backend action but unclear to customers.	Low.	Partial Panel support.	Replace with clearer `check_content_updates`.
`server_content_check_workshop_updates`	Keep internally, remove from customer dropdown for now	Useful once Workshop system is mature.	Low/medium.	Partial support.	Expose later as `check_workshop_updates`.
`server_content_install_updates_if_stopped`	Keep internally	Safe behavior for automatic content updates.	Low.	Panel support.	Customer label should be `Update content when stopped`.
`server_content_install_updates_next_restart`	Keep internally	Useful queued-update pattern.	Low.	Panel support.	Needs real next-restart integration.
`server_content_install_updates_now`	Keep advanced customer/admin	Updates while server may be running can break files.	Medium.	Partial support.	Gate by game support and require warning.
`server_content_install_updates_and_restart`	Keep advanced customer/admin	Very useful but needs locking and warnings.	Medium.	Partial support.	Should become `update_mods_and_restart`.
`server_content_update_workshop`	Remove from dropdown; keep as internal alias	Duplicate with Workshop update action.	Medium.	Partial support.	Hide until Workshop redesign is implemented.
`server_content_update_all`	Remove/merge	Duplicate with install/update all.	Medium.	Partial support.	Replace with one clear `update_all_content`.
`server_content_notify_updates_only`	Remove for now	Name suggests notification but no notification system exists.	Low.	Partial check-only path.	Reintroduce after notifications exist.
`server_content_validate_files`	Keep admin/advanced	Useful repair/validate action.	Medium.	Partial support via generic script action.	Rename to `validate_content_files`; game support required.
`server_content_backup_before_update`	Remove or redesign	Currently sets an option but there is no clear backup implementation in that path.	Medium due false confidence.	Incomplete.	Replace with first-class backup action and update workflow option.
Raw remote shell command	Admin only or remove from normal UI	Powerful but dangerous.	High.	Existing generic scheduler execution.	Should not be available to customers. Should be audited if kept.
Legacy `%ACTION=start`	Remove/deprecate	Current Panel uses API callbacks.	Low.	Agent support exists.	Keep only during migration if old task files contain it.
Legacy `%ACTION=stop`	Remove/deprecate	Same as above.	Low.	Agent support exists.	Migrate to action registry.
Legacy `%ACTION=restart`	Remove/deprecate	Same as above.	Low.	Agent support exists.	Migrate to action registry.

3. Competitor Feature Research

Sources reviewed:

Nitrado automated tasks guide: https://server.nitrado.net/guides/automated-tasks-en
Nitrado backup guide/FAQ: https://server.nitrado.net/en-US/guides/how-to-manage-and-restore-nitrado-server-backups/
BisectHosting Starbase schedules: https://help.bisecthosting.com/hc/en-us/articles/40101097661083-How-to-Schedule-Tasks-on-the-Starbase-panel
ZAP-Hosting scheduled tasks: https://zap-hosting.com/guides/docs/gameserver-scheduled-tasks
Pterodactyl client schedules API docs: https://pterodactyl-panel.mintlify.app/api/client/schedules
Shockbyte backups guide: https://shockbyte.com/help/knowledgebase/articles/how-to-backup-your-server-files
PingPerfect scheduled server messages guide: https://pingperfect.com/knowledgebase/706/7-Days-to-Die--Scheduled-Server-Messages.html
Host Havoc scheduled backups guide: https://hosthavoc.com/billing/knowledgebase/479/Creating-Scheduled-Backups.html
GTXGaming Rust restart warning task guide: https://help.gtxgaming.co.uk/en/article/how-to-setup-restart-tasks-with-messages-for-your-rust-server-15zwnyr/

Common commercial features:

Scheduled start/stop/restart.
Scheduled backups.
Scheduled console/RCON commands.
Restart warning messages.
Task offsets within a schedule.
Backup retention limits.
Restart-only-if-online option.
Manual and automatic backup creation.
Custom task scheduling.
Game-specific tasks such as Rust wipes or server message tools.

Notable competitor patterns:

Nitrado exposes simple automated power tasks: restart, start, stop. It also has automatic backups, but docs note timezone issues and game-specific backup behavior.
BisectHosting Starbase schedules support separate schedules and tasks, including power actions and command tasks with time offsets.
Pterodactyl's design is strong: schedules have multiple ordered tasks, time offsets, power actions, command actions, backup actions, only_when_online, and continue-on-failure behavior.
ZAP-Hosting exposes start/stop/restart, restart-if-online, create backup, and execute command, with rate limits.
Shockbyte emphasizes scheduled backup intervals and backup slot/auto-replace retention.
PingPerfect supports scheduled messages and Console/RCON commands for games like 7 Days to Die.
GTXGaming documents restart warnings/countdowns for Rust.

What GSP can do better:

Use typed, safe game-aware actions instead of raw commands.
Provide prebuilt restart workflows with save-world and warning steps.
Tie Workshop/mod updates into the Scheduler.
Add per-task locks and conflict prevention.
Add structured logs and visible success/failure.
Add notifications through Discord/email/panel.
Add game XML capability detection so users only see actions that work.
Add maintenance windows and "run when empty" automation.
Add resource-based triggers using existing status/resource collection work.

4. Recommended GSP Scheduler Actions

Customer-safe actions

Restart server.
Stop server.
Start server.
Backup server.
Backup selected folders.
Update Workshop mods.
Update server content.
Send warning message.
Run allowed RCON command.
Rotate logs.
Delete old logs using admin-defined retention limits.
Save world, if the game supports it.
Check server status.
Auto-restart if crashed.

Advanced customer actions

Scheduled wipe/reset for supported games.
Validate/repair server files.
Update SteamCMD game files.
Clone backup to another server.
Restore backup.
Update mods and restart.
Restart when player count is zero for X minutes.
Restart if memory too high for X minutes.
Restart if CPU stuck/high for X minutes.
Scheduled config file replacement from approved templates.
Scheduled database backup where applicable.

Admin-only actions

Arbitrary shell command.
Raw script execution.
Permission repair.
Force kill process/session.
Agent/node maintenance.
Cleanup storage outside a server home.
Clear global Workshop cache.
Repair file ownership.
Restart agent.
Reboot node.
Run panel update/maintenance.

5. Proposed Action System

Replace free-form action lists with a typed action registry.

Each action definition should include:

action_key
display_name
description
category
allowed_roles
required_permissions
supported_os
required_agent_capability
requires_game_running
requires_game_stopped
requires_rcon
requires_workshop_support
requires_steamcmd
arguments_schema
validation_rules
timeout_seconds
retry_policy
overlap_policy
conflict_group
log_policy
notification_events

Example:

scheduled_actions:
  restart_server:
    display_name: Restart Server
    role: customer
    agent_action: stop_wait_start
    required_permissions: [server.power.restart]
    args:
      warning_minutes:
        type: integer
        min: 0
        max: 60
        default: 5
      warning_message:
        type: string
        max_length: 160
        default: "Server restart in {minutes} minutes."
      save_world:
        type: boolean
        default: true
    timeout_seconds: 600
    conflict_group: server_power
    overlap_policy: skip_if_running

Recommended schedule model

Move from "one CRON line equals one command" to:

Schedule:
- name
- cron expression or interval
- timezone
- enabled
- only_when_online
- missed_run_policy
Tasks:
- ordered tasks within a schedule
- action key
- arguments
- time offset
- continue on failure

This matches the strongest commercial pattern and allows:

10 minutes before restart: send warning.
5 minutes before restart: save world.
At restart time: restart server.
5 minutes after restart: send Discord notification.

Suggested DB tables

gsp_schedules

id
home_id
remote_server_id
name
cron_minute
cron_hour
cron_day_of_month
cron_month
cron_day_of_week
timezone
enabled
only_when_online
missed_run_policy
max_runtime_seconds
created_by
created_at
updated_at

gsp_schedule_tasks

id
schedule_id
sort_order
time_offset_seconds
action_key
arguments_json
continue_on_failure
enabled
created_at
updated_at

gsp_schedule_runs

id
schedule_id
home_id
status
scheduled_for
started_at
finished_at
duration_seconds
trigger
last_error

gsp_schedule_task_runs

id
schedule_run_id
schedule_task_id
action_key
status
started_at
finished_at
exit_code
message
error
log_path
output_excerpt

6. XML Integration

Game XML should declare game-specific Scheduler support.

Example:

<scheduler_support>
  <action key="restart" enabled="1" />
  <action key="rcon_warning" enabled="1" />
  <action key="world_save" enabled="1">
    <command>save</command>
  </action>
  <action key="workshop_update" enabled="1" />
  <action key="wipe" enabled="1">
    <strategy>rust_wipe</strategy>
  </action>
</scheduler_support>

Global actions:

Start server.
Stop server.
Restart server.
Backup server files.
Rotate logs.
Delete old backups/logs.
Check status.

Game-specific actions:

Send RCON warning.
Save world.
Run console command.
Workshop update.
Mod update.
Wipe/reset.
Database backup.
Validate files.

Actions requiring RCON:

Warning message.
Save world.
Player-count-aware empty restart if query is not enough.
Allowed RCON command.
Game-specific graceful shutdown.

Actions requiring SteamCMD:

Update SteamCMD game files.
Validate/repair Steam game files.

Actions requiring Workshop support:

Update Workshop mods.
Repair Workshop mods.
Update mods and restart.

Actions requiring backup support:

Backup server.
Backup selected folders.
Restore backup.
Clone backup.

7. Agent Integration

Preferred direction

The agent should execute typed scheduled actions, not raw customer shell text.

New agent methods could be:

scheduler_action_start(home_id, action_manifest_json)
scheduler_action_status(home_id, action_run_id)
scheduler_action_log(home_id, action_run_id, offset)
scheduler_action_cancel(home_id, action_run_id)

The Panel should store schedules and send due actions to agents, or the agent should receive a structured schedule manifest from Panel. The cleanest long-term design is Panel-owned schedules plus an agent-side runner for actions.

Start/stop/restart

Agent should:

Use existing start/stop/restart functions.
Use the new agent status model as source of truth.
Wait for state transitions.
Return structured result.

Restart should:

Optional RCON warning.
Optional save-world.
Stop.
Wait configured seconds.
Start.
Poll until STARTING/ONLINE/UNRESPONSIVE.

Backup

Agent should:

Create compressed archives through a typed backup action.
Support include/exclude folders from safe config.
Store backup manifests.
Enforce retention.
Avoid backing up transient cache/log folders unless configured.

Update

Agent should:

Run SteamCMD update or server content update through typed job actions.
Avoid overlapping update with running backup/restart.
Mark restart required when applicable.

RCON/console command

Agent should:

Use existing send_rcon_command support.
Validate commands against action rules.
Log command and response.
Redact credentials.

Customer-safe RCON should use templates:

say {message}
save
save-all
game-specific warning command

Raw RCON text should be advanced/admin controlled.

Mod update

Agent should:

Run Workshop/server-content job runner from the Workshop design.
Return job status and logs.
Mark restart required.

Log cleanup

Agent should:

Delete only configured log paths.
Enforce age/size limits.
Log every removed path count/bytes.

Status/resource actions

Agent should:

Check process/session/port status.
Optionally check memory/CPU samples.
Execute conditional restart only after threshold duration.

Timeouts and failure reporting

Every action should have:

timeout
retry count
retry delay
result status
error message
log excerpt
correlation ID

8. Task Logs and User Feedback

Recommended run statuses:

pending
running
success
failed
skipped
canceled
timed_out

The UI should show:

schedule name
enabled/disabled
next run time
last run time
last status
last duration
current running task
output log
error message
retry count

Run details should show:

each task in the schedule
action arguments summary
start time
finish time
result
output/log

Do not rely only on scheduler.log.

9. Notifications

Supported notification channels:

Panel notification.
Email.
Discord webhook.
Generic webhook later.

Notification events:

Before restart.
After restart.
Backup succeeded.
Backup failed.
Update available.
Update installed.
Task skipped because server was offline/running.
Task failed.
Disk retention cleanup ran.

Security:

Webhook URLs must be stored securely.
Do not expose tokens in task logs.
Customers should not be able to send arbitrary webhooks from shared infrastructure unless allowed by policy.

Pre-restart warning types:

RCON in-game message.
Console command.
Discord/webhook message.
Panel notification.

10. Implementation Phases

Phase 1: Inventory/report only

Complete this report.
Do not modify code.

Phase 2: Remove or hide useless actions

Hide duplicate server-content actions from customer dropdown.
Keep internal aliases for backward compatibility.
Hide server_content_backup_before_update until real backup exists.
Keep raw remote command admin-only.

Phase 3: Safe action registry

Add PHP action registry.
Define roles, permissions, arguments, validation, and display names.
Replace hardcoded dropdown arrays.

Phase 4: Task logging

Add schedule/task/run tables.
Store run status and results.
Keep agent scheduler.log as low-level debug only.

Phase 5: Restart/backup/update actions

Implement typed restart with warning/save-world hooks.
Implement first-class server backup action.
Implement update server files action.

Phase 6: RCON warnings

Add game XML scheduler_support.
Add allowed warning/save commands.
Add command templates and validation.

Phase 7: Workshop update integration

Integrate with the redesigned Workshop/server-content job system.
Add update mods and update mods then restart workflows.

Phase 8: Notifications

Add panel notifications.
Add Discord webhook.
Add email.

Phase 9: Commercial polish

Multi-task schedules with offsets.
Clone schedule to another server.
Maintenance window mode.
Conditional empty-server restart.
Resource threshold triggers.
Missed-run handling.
Conflict and overlap visualization.

11. Final Recommendation

Remove or hide

Hide raw server-content internal actions from customer dropdown.
Remove customer-facing server_content_notify_updates_only until notifications exist.
Remove customer-facing server_content_backup_before_update until backup is real.
Merge duplicate update actions into clear labels.
Deprecate legacy %ACTION= task format after migration.

Keep

Start server.
Stop server.
Restart server.
SteamCMD update, renamed to Update server files.
Server content / Workshop update, once the Workshop system is mature.
Admin raw command only behind explicit admin permissions.

Build first

Typed action registry.
DB-backed schedules and run logs.
Restart server with warning and optional save-world.
Backup server with retention.
Update server files.
Update Workshop mods.
Notifications.

Admin-only

Shell command.
Raw script execution.
Force kill.
Permission repair.
Node cleanup.
Agent restart/reboot.

Delay until later

Resource-triggered restarts.
Wipe/reset workflows.
Restore backup scheduling.
Clone schedules.
Generic webhooks.
Advanced conditional schedules.

Summary

The current GSP Scheduler is functional but primitive. It stores CRON lines on agents, executes shell commands, and often calls back into the Panel through wget. That makes it flexible, but it does not provide the safety, visibility, or polish expected from a modern commercial game hosting panel.

The recommended path is a typed, DB-backed schedule system with safe action definitions, game XML capability flags, agent-side action execution, structured run logs, notifications, and first-class workflows for restart, backup, update, Workshop mods, and RCON warnings.

25 KiB Raw Blame History

GSP Scheduler Actions Design

Scope

1. Current Scheduler Module Findings

Files inspected

Current database tables used

Current actions

How tasks are created

How tasks execute

How task results are logged

Current limitations and bugs

2. Current Action Review

3. Competitor Feature Research

4. Recommended GSP Scheduler Actions

Customer-safe actions

Advanced customer actions

Admin-only actions

5. Proposed Action System

Recommended schedule model

Suggested DB tables

6. XML Integration

7. Agent Integration

Preferred direction

Start/stop/restart

Backup

Update

RCON/console command

Mod update

Log cleanup

Status/resource actions

Timeouts and failure reporting

8. Task Logs and User Feedback

9. Notifications

10. Implementation Phases

Phase 1: Inventory/report only

Phase 2: Remove or hide useless actions

Phase 3: Safe action registry

Phase 4: Task logging

Phase 5: Restart/backup/update actions

Phase 6: RCON warnings

Phase 7: Workshop update integration

Phase 8: Notifications

Phase 9: Commercial polish

11. Final Recommendation

Remove or hide

Keep

Build first

Admin-only

Delay until later

Summary

25 KiB

Raw Blame History