web scraping tool working with moduls for scrape different sites to different DB

Hi, for the last several years, we have been working on scraping, extraction and crawling websites to extract and retrieve specific attributes and aggregation from various unstructured data, websites and have assembled and synchronized to databases and datastores before and have used Scrapy framework for some of our work. In particular, we have extracted information from Groupon, Wikipedia, Youtube and other product based web sites for very specific attributes. Please find below our short experience summary. * Have several years experience developing Text Mining and Information Extraction and Analytics for web crawling, scraping, extraction and aggregation from unstructured big data such as web-pages and text corpus, assembling, synchronizing and populating them into databases, datastores and search-indexes(Lucene, Solr) for analysis, search, reporting and dashboard. * Extensive experience using Perl, PHP, Python, C, Java, .NET with MySql, Oracle, MS-SQL Server * Information Extraction Tools : Scrapy Framework, Weka, R, Excel, Perl-CPAN Packages for Extraction. Estimated Budget : ~ 245 USD ( Timeline : 5-7 days ) Price,milestones and timelines flexible and negotiable based on exact project specifications and details or for any additional project work. Would you be interested in sharing more information regarding your project.

I have +10 years of experience in programming. I am expert in web scraping. I can easily make what you want for you to be general for any type of website if you are interested. Please let's discuss more about details when you have time. Thanks, Helmot

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi

A proposal has not yet been provided

A proposal has not yet been provided

Hello! I'm a computer engineer with more than 12 years of experience in software development. My suggestion is to build a PHP or even Bash script to get data from the websites, then storing all data on a PostgreSQL or MySQL database. This will ease the process of updating/adding new records. I always deliver all source code generated and make myself available for questions and answers, so that you'll be aware of every part of the work. No hidden stuff. Please check my profile to see I have done many jobs similar to this one lately, so you have more chances to have the work done in less time by contracting me. Can you specify what kind of information you need from the websites? Or do you need to get all contents? Keep in touch so we can talk about the project and even about the costs, if needed. I'm ready to start working right now. Best wishes. Alex.

Hi, Your project sounds very interesting and I'd really love to work on it. As for my experience, you can see for yourself that I've worked here mainly as a content and article writer; but, I've also done some VA work, which included scraping using a custom made software. Recently I've created a program for scraping Freelancer site in order to make bidding easier. You're welcome to check my reviews and should you have any questions, I'll be happy to answer. Kindest regards. Milan

I can do that for you in Java (console program), it will run on Linux/Unix, no problem What kind of database do you use? What kind of data needs to be scraped and compared against DB?

Hello, the task seems obvious, I would like to do it for you. But, of course I would need more detailed information about your requirements. Please contact me if you are interested. Short info about me: I am a senior web developer from Cologne, Germany, with 15 years experience on lage projects. My focus is mainly on design of big backend database structures, search facilities, data transfer processes, QA and performance tuning. Currently I am free and looking for new challenges. Best regards, Viktor Gotwig.