I’ve written a script that monitors stock prices, but I don't want to keep my laptop open all night. What are the best ways to host and schedule a Python script to run every hour? Should I look into Cron jobs on a VPS, or are there easier "Serverless" options like AWS Lambda?
3 answers
For a simple script, AWS Lambda is excellent because of the "Free Tier." You can use EventBridge to trigger your Python function on a schedule. However, if your script runs for more than 15 minutes or needs a lot of RAM, a small VPS (like DigitalOcean or AWS EC2) is better. On a Linux VPS, you just use crontab -e to schedule your task. It’s highly reliable and gives you full control over the environment. For advanced users, Dockerizing the script and running it on a container service ensures it works exactly the same as it did on your local machine.
When running scripts 24/7 on a VPS, what’s the best way to handle logs so you can check if a script failed while you were asleep without the log file getting too huge?
If you want zero setup, try "GitHub Actions." You can schedule scripts to run on GitHub's servers for free as long as the repository is public and the task is light.
That’s a great "hack," Mary! I use GitHub Actions for all my small web scrapers. It’s incredibly easy to set up with a simple YAML file.
Steven, you should use the logging module in Python with a RotatingFileHandler. This allows you to set a maximum file size for your logs. Once it hits say, 5MB, it automatically starts a new file and deletes the oldest one. This prevents your server from running out of disk space. For even better visibility, I often have my scripts send a quick message to a private Slack or Discord channel if they encounter a critical error during execution.