Ruby (on Rails) expert for Web Scraper / Web Scraping Job needed

Stängt Publicerat 4 år sedan Betalades vid leverans
Stängt Betalades vid leverans

Intro.

I am looking for a multi-talented Ruby (on Rails) expert in web scraping. You are going to build an entire webapplication with Ruby (on Rails) - using the newest Rails 6.0.0 version and the latest compatible Ruby version. You will find all the links to Frameworks, Templates, Gems etc. below.

Target website.

The first website will be AirBnB's room / housing listings. Please take a very careful look at a few of those listings. AirBnB's is randomly generating id and name HTML-attributes to protect their site from scraping. The structure seems to stay the same though. You will always be presented with 300 search results - even in cities like New York City. To see all listings you have to adjust the price (increase, decrease). In addition to that, use the map to get all the results (zoom in and out, move the map).

Backend.

• You will use the Kimura framework for scraping jobs (link below)

• You will create a "command script" that triggers an AWS EC2 server to be setup, configured and destroyed for every single scraping job (see tutorial / links below). This server will function as a proxy. You are going to use Terraform for that.

• Authorization through CanCanCan (Guest, User, Manager, Admin)

• Authentication through Devise (Basic User table with login is enough)

• Historization of all listings through Papertrail

Scraping job.

The scraping job needs to run when triggered (e.g. if I want to have a new city to be scraped). Also, the job needs to re-scrape an existing listing - the old version can be saved through Papertrail, or you just create a new listing and reference the old with the new one.

Frontend.

I will provide you with a Bootstrap admin template that needs to be used for the frontend. This Bootstrap admin template is built on top of the Bootstrap framework, SASS, jQuery, HTML5, and CSS. So you need to know how to work with those!

Links.

[login to view URL] (Ruby Framework)

[login to view URL] (Create AWS servers tutorial)

[login to view URL] (EC2 proxies)

[login to view URL]

[login to view URL]

[login to view URL]

Amazon Web Services CSS HTML5 PostgreSQL Ruby on Rails

Projekt-id: #20015968

About the project

11 offerter Distansprojekt Aktivt 4 år sedan

11 frilansare har lagt bud på i genomsnitt $523 för det här jobbet

kipdev13

Nice to meet you. I've read the task description and can develop definitely. I'll provide you with the highly skill to complete this task in time. I'll also do my best for you. Please contact with me. Thank you ve Mer

$555 USD inom 10 dagar
(12 omdömen)
5.0
zeke

Dear Customer! I have built many scrapers. This is my favorite type of job. I am absolutely confident I can complete your project easily and on time. Ready to begin immediately and finish as soon as possible. Please co Mer

$500 USD inom 14 dagar
(17 omdömen)
5.5
sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Mer

$500 USD inom 7 dagar
(7 omdömen)
4.6