Scrape and Parse a Lyrics Website

$250-750 USD

Avslutat

Publicerad

över nio år sedan

$250-750 USD

Betalning vid leverans

IMPORTANT! I will not respond to any form letters or lists of projects that you've done in the past. What I specifically want to see from any proposals is your approach to the problem. Anything other than that will be ignored. I'm looking for someone to write a script to scrape a lyrics website and parse it into the following three tables: SONGS TABLE id Artist - Artist whose song it is Song Title - Title of the song Featured Artists (comma-separated) - All the featured artists on the song Number of Verses - Number of Verses in the song Verse IDs - ids of the verses from the verse table in order of appearance on the song Album - Name of the Album the song Appears on VERSE TABLE id artist - Artist name verse - Text of the verse order - Order number of the verse in the song ALBUMS TABLE id artist song IDs - list of song ids in order that appear on the album The site in question is a combination of text files and web pages. It has formatting, to delineate these data points, but it can be inconsistent. Your deliverables will be: - The database in MySQL format - A script that crawls and scrapes the site and updates the database whenever new lyrics are added. It should be smart enough to crawl the new lyrics page and identify what hasn't been downloaded in the past. IN YOUR PROPOSAL - Let me know how you might approach this project and I'll respond with the website in question. The person with the best approach will be chosen.

MySQL

PHP

Python

Project ID: 6995767

Om projektet

37 anbud

Distansprojekt

Senaste aktivitet nio år sedan

Ute efter att tjäna lite pengar?

E-postadress

Fördelar med att lägga anbud hos Freelancer

Ange budget och tidsram

Få betalt för ditt arbete

Beskriv ditt förslag

Det är gratis att registrera sig och att lägga anbud på uppdrag

37 frilansar lägger i genomsnitt anbud på $480 USD för detta uppdrag

@goutam08

Hi, Here is Techno Exponent (11 years Exp.) reputed Web Development Company with all 127 positive reviews in freelancer.com. We have 53 Experienced Employees, 500+ happy clients. Techno Exponent is the Drupal Association Member. We are very interested to execute your Project in THE MOST SUCCESSFUL way! *****We provide 6 months Free Bug Fixing Support***** PROJECT SCOPE: We will Scrape the lyrics from your ref. website, Could you give me the URL from where you need to scrap! I have understood your Concept of SONGS TABLE, VERSE TABLE AND ALBUMS TABLE. What is the Most Important thing is your reference site URL! Please provide it to me! You are requested to initiate the chat with us! Our next Step will be to develop: >Project Development Plan >Milestone Break up Document. Please check this link: http://www.freelancer.com/u/goutam08.html *No Negative Review. *No “Incomplete” Tag. Freelancer.com Exams: >Passed PHP exam with 99% >Passed UK English exam with 88% >Passed US English exam with 81% Thanks. Jyoen T.

$701 USD Om 36 dagar

5,0

(198 omdömen)

9,0

@mantislin

Hi sir, I am scraping expert, please check my feedback then you will know. Can you tell the Lyrics website then I can check more details? Thanks, Kimi

$722 USD Om 6 dagar

5,0

(421 omdömen)

8,2

@entrepreneur2000

Hello, First of all, I would suggest a bit different structure of MySQL database as what you proposed is not really the most efficient from architecture perspective. So I would: - Remove Verse IDs from Songs table and add Song ID column (foreign key) to Verse table - Remove numer of verses from Songs table as this number can be counted dynamically and doesn't need to be stored - Add Album ID to Songs table and remove song IDs from Album table This setup will ensure that queries against the database are run in efficient way. Now in terms of the scraper I would implement a console PHP script which connects to the website in questions and scrapes data by going through index of songs and dowloading each one of them. The main problem here is how to retrieve the new added songs. The simpliest approach is to go through the website index once again and check whether each page was already scraped or not. But I need to check the website first to confirm the best approach here. Can you please let me know the lyrics website and contact me to discuss the requirements of this project in more detail? My background: I am a professional web developer with more than 7 years of extensive hands-on experience in PHP, HTML, CSS, Java, JavaScript, JQuery, MySQL and other web technologies. I look forward to working with you on this project. Thanks, Alexey

$669 USD Om 7 dagar

5,0

(171 omdömen)

8,2

@chirgeo

Hi. Well, what can I say about the approach, all depends on the site itself. The issue in such projects is not the DB or parsing some data. Usually the issue is with limits of requests you can make. So, choosing a random proxy from list on each request is a good approach from the start. So, I guess I need first to take a look at site and then we can discuss more about it. Thx, Hope we will collaborate.

$555 USD Om 10 dagar

5,0

(144 omdömen)

7,9

@srinichal

I will be using PHP CURL along with regex to parse the various formats and then dump the same in to DB

$526 USD Om 10 dagar

4,8

(145 omdömen)

7,4

@hwanghendra

Hi I believe this can be done using PHP & curl. To be able to say for certain how to crawl the new lyrics page, I will have to check the website first.

$500 USD Om 5 dagar

4,9

(412 omdömen)

7,4

@mobileappdevin

The Scrape approach as follows: Technology: * nodejs (using cheerio plugin) * datastore mysql * scrape manager: Django (python) Approach: Initially when scraping the Songs Table with pagination, the script would get songs details url. While visiting songs details url we can get album, artist etc info. Next steps would be visiting details for those entities to get the details info to be scrapped. On the song details page we need to find a way to download the music file if available depending on the website. It would be great if we get the name of the website to create a more detailed and specific approach, as the above one is very general. Waiting for your response. We have done similar jobs as well on products & image domains. If you permit we can show you as well. Thanks & Regards.

$611 USD Om 15 dagar

4,9

(45 omdömen)

7,0

@prashushinde9

Hi, I (Myself a senior PHP, ASP.NET programmer using C# and VB) have a team having experience of more than 10 years in developing web and mobile applications. I have a team of expert programmers who write robust and bug-free codes at the very first instance. After seeing our quality of work and the cooperation offered to you even after the work has been delivered to you, you’ll come to us again and again to avail our services. I read through the job details extremely carefully and I am absolutely sure that I can do the project very well. I can achieve the results that you are asking for within your timeframe and budget. I am confident I can exceed your expectations and I would love to work on your project. Thanks, Priyanka

$773 USD Om 15 dagar

4,8

(54 omdömen)

7,1

@eLYTeam

Hello Sir, First of all, thanks in advance for your atention. Wondering if you could disclose the lyrics website so I could take a look at it. Also, the approach is simple. The are several tools out there to do this kind of job. Among them: Selenium & PhantomJS are the most populars. I would use any of them (or another suggestion) to do the job. Looking forward to know about you, Wilson.

$500 USD Om 7 dagar

4,8

(79 omdömen)

6,6

@dusmanija

Hello Sir, I am expert in web scraping and let me just stick to the point like you said. I can deliver the data in mySql as I have a lot of experience with it. I need to see the site in order to determine level of inconsistency. Basically it will be a java scraper that will scrape all the pages variations (because you said there are variations of pages). It will remember what has been scraped so far by song and artist combination or song url and will not go to that page again when doing the scrape again (smart scraping). Script will be made so it can be run via cron whenever you set it to start. We can talk more about the subject so please contact me. In order to tell more I need to se the site first. Check my profile to see that people who worked with me are extremely satisfied with results and speed. I have 100% completion rate and I can start right away. Best regards, Dusan

$495 USD Om 6 dagar

5,0

(50 omdömen)

6,0

@anuyadav1

A proposal has not yet been provided

$555 USD Om 10 dagar

4,8

(84 omdömen)

6,2

@sashamd

Hi, I would like to write this script using Ruby or PHP. Please contact me if you have any questions. Best wishes, Sasha

$300 USD Om 5 dagar

5,0

(30 omdömen)

5,8

@srikartiram

Hi, I'm able to handle this task perfectly. I'm able to do this in Perl program, which you can run from your desktop by double clicking on it. I will scrap and upload the data's into MySQL then from next time when you run the script it would automatically upload into that existing table. Please provide me the website, i will start the progress straight away. Awaiting for your most precious reply. Thank you Karthik:)

$300 USD Om 4 dagar

5,0

(39 omdömen)

5,2

@kotharigaurav90

Hello Sir , I understand your all requirements , Can We Discuss About The Project ? please check your private memorandum.

$333 USD Om 10 dagar

4,6

(49 omdömen)

5,8

@omprakash001

Hi, I hope you are doing well. I have read requirement and i am interested in this job. My Approach : - 1) First i will make database for the website (system). 2) Design mock up (World Class design) . 3) Convert It in the HTML5/css3 with responsive layout. 4) Make admin panel so that admin user can manage whole website. 5) Development of the whole front -end I hope we will meet soon for further discussion Thanks a lot OM

$526 USD Om 20 dagar

4,9

(19 omdömen)

4,9

@dsrrathor

Hello, My approach for the scrapping: 1. I'll write a python script that will download the indexes for song from site. I believe site will have some kind of index (as most site have) like alphabetical order etc 2. Whenever the scapper will run, it will first download the indexes and match the indexes with those stored in database. 3. If new indexes appear, it will go to that link and scrap the data in the format demanded by you and update them in database and also update index in database 4. Scrapper can be run on daily/weekly basis to see if new lyrics are available P.S.: As you haven't mentioned the site, so the approach might slightly differ with the site as indexes pattern might be different.

$333 USD Om 5 dagar

5,0

(20 omdömen)

4,7

@sknadeem123

A proposal has not yet been provided

$277 USD Om 3 dagar

5,0

(24 omdömen)

4,8

@prabakarm23

Greetings for the Day!!! I have 6 years of experience in .NET, VBA Macros, VB script,PS Script and VB creation with application like SAP, Internet explorer, Microsoft Outlook,PDF& Text files, MS Access and SQL Server databases. And also I have worked on extraction with websites like Amazon, Cellpex,Costco,etc., hope if awarded with this project I can make it best and better with maximum 100% accuracy and satisfaction. please award me this project and contact me for further details Thanks Prabakar M

$277 USD Om 3 dagar

5,0

(23 omdömen)

4,4

@klimzpp

Hello! Can work on it together with this project - https://www.freelancer.com/projects/php-Software-Architecture/Pull-Music-Metadata-into-from.html

$277 USD Om 4 dagar

5,0

(8 omdömen)

2,9

@webeduzitsols

Webeduz is a leading solution provider for Internet based applications. Established in 1999, The Company has been promoted by some highly experienced Professionals dedicated to provide total IT solutions under one roof. It possesses not only the latest technology gadgets but also the most knowledgeable and experience hands to offer most user friendly customized solutions. Webeduz Net provides high quality on site services for software development and the end users on a broad range of hardware & software platforms and latest technologies. Within the first year of its operations, Webeduz has carved a niche for itself in the IT industry and has increased its business by acquiring some major domestic projects. No doubt the company has been able to make a name for itself in a relatively short span of time only because of its ability and commitments to ensure customer satisfaction by rewarding quality work on right time and in a right manner.

$394 USD Om 11 dagar