Commit Graph

6 Commits

Author SHA1 Message Date
Ross 9c86b32c3b . 2025-10-20 21:15:33 +01:00
Ross 2f8df5f820 Add comprehensive articles on Vascular Dementia and Wallerian Degeneration
- Created a detailed article for Vascular Dementia covering key facts, terminology, imaging findings, differential diagnoses, pathology, clinical issues, and diagnostic checklist.
- Developed an extensive article on Wallerian Degeneration including key facts, terminology, imaging features, differential diagnoses, pathology, clinical issues, and diagnostic checklist.
2025-10-17 22:11:27 +01:00
Ross adc21f5795 feat: Implement passive XHR/fetch capture using Playwright
- Added `capture_passive_playwright.py` for synchronous passive capture of XHR/fetch responses.
- Added `capture_passive_playwright_async.py` for asynchronous passive capture using Playwright's async API.
- Introduced `save_page_snapshots.py` to save full page HTML snapshots with deduplication.
- Removed obsolete CSRF token capture JSON file.
2025-10-14 19:48:01 +01:00
Ross e3d081cbff Add scripts for capturing network responses and extracting document content
- Implement capture_with_cdp.py to capture all network response bodies using Chrome DevTools Protocol.
- Create extract_sections.py to extract sections from STATdx snapshot HTML files into JSON Lines.
- Add unpack_document_content.py to extract `documentHtml` from captured JSON bodies and save as HTML files.
- Update .gitignore to include playwright_profile.
2025-10-14 17:44:20 +01:00
Ross fd5922b3a6 Update .gitignore and requirements; enhance logging in Playwright scraper 2025-10-14 13:56:47 +01:00
Ross 491fca2f42 Add Playwright-based scraping and login scripts for STATdx
- Implemented `statdx_play_chrome.py` to launch Chrome via Playwright, allowing for headful browsing and capturing XHR requests.
- Created `statdx_playwright.py` for headless scraping of the STATdx application, including automatic login and HTML saving.
- Developed `statdx_requests.py` for form-based login and scraping, providing a fallback for sites using traditional authentication.
- Added example HTML output from Playwright scraping to `topics_playwright.html`.
- Captured XHR request data in JSON format for analysis in `xhr_captured/app.statdx.com_csrf_token_8000d0ed_20251014T124122Z.json`.
2025-10-14 13:46:39 +01:00