Find Jobs
Hire Freelancers

Web scrape of 5-7 online retailers using Ruby Mechanize/Watir/Nokogiri (repost)

$25-50 AUD / hour

Imeghairiwa
Imechapishwa over 6 years ago

$25-50 AUD / hour

I run a data analytics consultancy in Sydney, Australia. I have a requirement to setup weekly scrapes of at least 5 large (50,000+ SKUs) online retail websites. The project will be split to scrape a single website to start with, extending to the others assuming satisfaction with the work completed. Technical requirements: - Scrapes must be written in Ruby using Mechanize/Nokogiri or Watir as appropriate - Where possible, avoiding JS and scrape via direct HTTP requests using Mechanize - i.e requires strong skills in tracing/debugging HTTP requests/responses - Alternatively, load page using Watir webdriver then pass off cookies to Mechanize - Last resort would be brute force Watir webdriver approach (scroll, point, click, etc) - Implement utilities for use of proxy / IP rotation where required (access to pool of anonymous IP addresses will be provided) - Scheduling via cron job to run at least weekly - Must use object oriented approach and design in a DRY manner so that code can be leveraged for future scrapes of additional sites and ensure future code manageability - Must use Activerecord to persist records scraped - Must implement logging (production logging to file + verbose debug option to STDOUT) Personal requirements: - Must be a great communicator both in terms of coding concepts and project management - Good at clearly laying out the workplan, milestones and progress - Must be available for 2x daily stand-up calls with Sydney AUS (timing TBC at mutually agreed time) - Must be able to demonstrate prior experience with ruby web scrape projects similar to above - Code must be well-structured and well-documented / commented Opportunity for ongoing Ruby backend and RoR web app build if this project is a success.
Kitambulisho cha mradi: 15865779

Kuhusu mradi

19 mapendekezo
Mradi wa mbali
Inatumika 6 yrs ago

Unatafuta kupata pesa?

Faida za kutoa zabuni kwenye Freelancer

Weka bajeti yako na muda uliopangwa
Pata malipo kwa kazi yako
Eleza pendekezo lako
Ni bure kujiandikisha na kutoa zabuni kwa kazi
19 wafanyakazi huru wana zabuni kwa wastani $37 AUD/saa kwa kazi hii
Picha ya Mtumiaji
As the TOP 5th freelancer of this site, I can deliver the good result in time. Developed some kind of data scraping using Ruby on Rails and PHP. With the wonderful experience and enough knowledge, I can make the success of your site. And also can work on the RoR back-end / front-end Looking forward to working with you.
$35 AUD ndani ya siku 60
4.9 (18 hakiki)
7.8
7.8
Picha ya Mtumiaji
Hello,sir I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert and smart technic with web scraping (1000 over projects for scraping), automation and bot(500 over projects for automation and bot), selenium, mobile (android, ios native and others). Especially, I am an expert in betting project and gambling games and sites(bet365, bwin, pinnacle, betfair...) ,,, shopping auto buyer(adidas, footlocker, eastbay, supreme and etc) So, I can solve the captcha automatically. Please contact me, then we can discuss about the details and the price. And I am free in recent days, so I can do my best for your project. Thanks.
$41 AUD ndani ya siku 40
4.9 (91 hakiki)
6.9
6.9
Picha ya Mtumiaji
I've special skills in web scraping using Python and Ruby. Generally use direct HTTP requests, but if website asks javascript running on client, no problem by using selenium and webdriver. Have scraped on-demand, social, real-estate websites and so on. Please feel free to discuss with me more about this job. Best
$44 AUD ndani ya siku 40
5.0 (11 hakiki)
5.8
5.8
Picha ya Mtumiaji
I have good experience for web scrap app development. I can setup an app that scrapes at leat 5 large online retail websites. Let's discuss more in PM.
$39 AUD ndani ya siku 40
5.0 (6 hakiki)
5.6
5.6
Picha ya Mtumiaji
We have worked on the large scale scrapers which work in the same way as you described in the project requirements. Are you available to discuss the details now? Talk soon, Zoran
$40 AUD ndani ya siku 40
5.0 (2 hakiki)
5.3
5.3
Picha ya Mtumiaji
Hello, I am scraping expert, I have did too many similar projects, Please check my feedback then you will know. Can you tell me more details? Then I will provide demo data for you.
$25 AUD ndani ya siku 40
4.9 (60 hakiki)
5.5
5.5
Picha ya Mtumiaji
Hi there, I have read everything and our team is very experienced in Ruby on Rails, so please share complete requirements or let me know from which websites, we have to scrap the data? Do you want to display the data on web page or store in the database or excel file? Regards, YK LEADconcept
$38 AUD ndani ya siku 40
5.0 (2 hakiki)
5.6
5.6
Picha ya Mtumiaji
Hi, I’m a Web Designer/Developer from the UK. My name is Mike. Your project description sounds interesting to me and I do have skills & experience that are required to complete this project. Let's have a quick chat when you're online.
$41 AUD ndani ya siku 40
5.0 (1 hakiki)
3.2
3.2
Picha ya Mtumiaji
Hello!, do you need a script or my work? I have scraped Amazon 150000 URL (Title, Price, Bullets, Pictures). look please at my portfolio.
$41 AUD ndani ya siku 40
5.0 (2 hakiki)
2.2
2.2
Picha ya Mtumiaji
Hey, how is it going today? My name is Michael, I'm a web designer/developer from Boston. I've had a look at your project description and feel that my skills match your requirements perfectly. In fact, I did a very similar project recently which I can share with you. Let me know when you have a minute to discuss.
$41 AUD ndani ya siku 40
0.0 (0 hakiki)
0.0
0.0
Picha ya Mtumiaji
Hey there, I will develop a web scraper for online retailers using Ruby in combination with Watir. I will implement advance proxy rotation in order to achieve maximum performance. I have a very complex scraping projects behind my back - the last one was scraping 11 huge listing websites updating the database every hour. Are you available for a chat to discuss the project? Thanks, Lazar
$37 AUD ndani ya siku 40
0.0 (0 hakiki)
0.0
0.0

Kuhusu mteja

Bedera ya AUSTRALIA
Maroubra, Australia
5.0
3
Njia ya malipo imethibitishwa
Mwanachama tangu Jun 2, 2017

Uthibitishaji wa Mteja

Asante! Tumekutumia kiungo cha kudai mkopo wako bila malipo kwa barua pepe.
Hitilafu fulani imetokea wakati wa kutuma barua pepe yako. Tafadhali jaribu tena.
Watumiaji Waliosajiliwa Jumla ya Kazi Zilizochapishwa
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Onyesho la kukagua linapakia
Ruhusa imetolewa kwa Uwekaji wa Kijiografia.
Muda wako wa kuingia umeisha na umetoka nje. Tafadhali ingia tena.