web crawler with javascript support

I need a webcrawler library which is supposed to run in the backend/docker with some JavaScript support. There are many websites which are inserting some JavaScript checks in their page and I want them to pass What it needs to support: 1. I need a way to send standard HTTP requests with an URL and some parameters (POST/GET/PUT, etc). At best, I would like this call _NOT_ to be asynchronous. 2. A way to store cookies from request to another, like a cookie jar; 3. I need to get the raw body of the request - the way server responded to the client; 4. I need also a formatted HTML (tidy/*) of how the document actually looks like. I would like the bidders to start their bidding post with "+++". This is how I can tell if you actually read the post. I will simply ignore/remove the others - there are too many to read and many of them are nothing but automated posts/spam. The language you choose is not important as long as it respects the things above. I would prefer Java or Python, but I can accept others - just let me know about this in your post. Again, this will be running in an linux docker, so be prepared for this. I am a programmer myself and I know this can be done. I did it in the past, meanwhile technologies changed and it may take a while until I am up to date with everything. This is the reason why I need your help. Also, this means I have a fair idea of the effort needed, so don't go wild with your bid here. Deliverables: * full source code; * a small demo in docker on how you will crawl [login to view URL] - 4-5 pages at your choice, no need to interpret the results;