Multi-Market Trading Data Scraper

I’m building a backtesting engine and now need a robust scraping solution that feeds it three streams of information: 1. Clean, timestamp-aligned OHLCV data. 2. News, social-media, or other sentiment signals tied to the same symbols and dates. 3. Company financials, earnings releases, and key macro-economic numbers. Coverage must span US stocks and ETFs, cryptocurrencies, and major forex, commodity, and futures contracts. All material should come from freely accessible public websites or APIs you select, so part of the job is choosing reliable sources that respect their own terms of service. Here’s what I expect at hand-off: • Python (preferred) scripts or notebooks that automate the collection, parsing, and normalization of each data type. • A lightweight scheduler or CLI that lets me pull fresh data on demand or run daily batches. • Output delivered as CSV files or a simple SQLite/PostgreSQL database, with consistent symbol naming and UTC timestamps so I can drop it straight into my backtester. • Clear documentation explaining the data sources, any rate-limit handling, and how to extend the scraper when new tickers or markets are added. • A brief demo run showing sample results for one equity, one crypto pair, and one futures contract. Please use well-supported libraries (requests, BeautifulSoup, Selenium, Scrapy, pandas, etc.) and keep external dependencies minimal. Error handling, retries, and polite throttling are important—this must run unattended without breaking site rules. I’ll review the code for readability, rerun the demo, and validate that fields line up across datasets. Once everything passes those checks, the project is complete.