Extract data via JSON API and make reports

Imekamilika Ilichapishwa Jun 4, 2015 Kulipwa wakati wa kujifungua

$1500-3000 USD

Kulipwa wakati wa kujifungua

Imekamilika Kulipwa wakati wa kujifungua

We need to build a robust web application in PHP and mysql that will do the following:

1.- Data extraction and saving:

1A.- Connect via JSON API to a popular website that lists products in a category to sell and extract several variables.

1A-1.- Connect to an URL in the format: www.URL.com/category=CAT1&limit=200&offset=0 where CAT1 is dynamic category code of products(to be explained later on this document), limit is the allowed max products to display per URL request, offset is the start of the listing or paging.

1A-2.- Extract and save several variables from each row of the product listing to the database: ProductID, ITEM-NAME, Price, QTY-SOLD, location, SellerID, Product-URL. 200 products info should be saved here.

1A-3.- Connect to the same URL now with Offset=201 and repeat steps X amount of times to be defined later on.

1A-4.- When X amount of times has been repeated on the same category, step 1 will be repeated now with a new category. Steps 1 to 4 will be repeated until all categories with x amount of times are saved.

1B.- Connect via JSON API to the seller page to extract the Seller Name

1B-1.- IF NickName of a SellerID is not known, then:

1B-2.- Connect to URL [login to view URL] and extract and save variable NICKNAME.

2.- Assign Internal-ID:

2A.- All item-ID should be assigned an internal-ID. When an itemID expires, it is republished with a new itemID. We need to track and know that a new itemid is in fact just the old item republished. With the parent-item-id variable we can know this information.

2A-1.- When a new ITEMID is detected, connect to item URL [login to view URL] and look for parent_item_id and store variable.

2A-2.- If parent_item_id is equal to another itemID then assign Internal-ID of previous ITEMID, else assign new internal-ID.

2B.- We need to adjust for QTY-SOLD from the old to new itemID when a new itemID is assigned to an existing internal-ID

2B-1.- Connect to the old item ID page. [login to view URL] where ITEMID is the old item ID.

2B-2.- Store the same variables as in step 1A-2

Note: What we are doing here is the following: Suppose we recorded the variables of ITEMID1 on June 15. The ITEMID someone expired on June17 and we detected the republished new ITEMD2 on June 20. We need to know the item QTY-SOLD on June 17 since its an unknown.

Note2: all of process numer 2 should only be done after the initial loading or starting phase. It is not needed to run on the first time.

3.- Data extraction Screen and automation.

A.- Thread Status list

3A-1.- There should be a screen where the status and URL of the current data extraction thread is displayed. There should be an option to pause, stop, start the thread. Diferent URL types should be on different threads like categories or sellerid or itemid URLs.

3A-2.- The app should have the ability to configure and run multiple threads of data extraction and also the ability to throttle them by amount of URLs per minute per thread.

3A-3.- The app should be able to be run manually or automated by X amount of time like once a week, once a month, etc. by any method desired either crons or another method.

3A-4.- Particular attention should be devoted to anticipate any errors and display them accordingly to take corrective action.

There are additional points, please look at attached document. Thanks

Basically this is a PHP app with mysql that connects via json api to a popular site

Time frame:

1 month

PHP Software Architecture

Kitambulisho cha Mradi: #7802076

Kuhusu mradi

35 mapendekezo Mradi wa mbali Ipo mtandaoni %project.latestActivity_relativeTime|badilisha%

Unapania kutengeneza pesa?

Faida za kutoa zabuni kwenye Freelancer

Weka bajeti yako na muda
Lipwa kwa kazi yako
Onyesha ombi lako
Ni bure kujisajili na kuweka zabuni kwa kazi

Imetuzwa kwa:

Picha ya Profaili ya UtopiaInt
UtopiaInt

I have reviewed your bid description and your attached document. We have extensive experience in building similar data crawler and extraction systems. We have implemented a few for our different clients for differen Zaidi

$3000 USD kwa siku 80
(Maoni 35)
8.4

35 wafanyakazi huru wanazabuni wastani wa $3343 kwa kazi hii

Picha ya Profaili ya sanjay2004
sanjay2004

A proposal has not yet been provided

$5882 USD kwa siku 60
(Maoni 715)
9.9
Picha ya Profaili ya leadconcept
leadconcept

Greetings, I have read Project Description & fully understood your requirements but I am afraid that time-frame of 1 month and your budget is low, so would you be flexible in it? Please drop me a message thru' PMB t Zaidi

$7647 USD kwa siku 60
(Maoni 112)
9.2
Picha ya Profaili ya cr8tivewebexpert
cr8tivewebexpert

Hi, We have reviewed your requirements for building this multi threaded automated system. We are interested coz we have worked on such data extraction solutions and automation. Thank You

$5882 USD kwa siku 45
(Maoni 305)
9.1
Picha ya Profaili ya jatindhoot
jatindhoot

Hello, I have gone through the attached doc properly. Are you adamant on PHP to be used? As PHP does not support multi threading in true sense I would recommend .NET Can we use MongoDB for increased performanc Zaidi

$1764 USD kwa siku 30
(Maoni 216)
9.3
Picha ya Profaili ya e3d
e3d

I can do this, no problem. I'll dive deeper in your doc file (don't have ms word right now) and will be ready to discuss this tomorrow. Thanks .

$3823 USD kwa siku 30
(Maoni 303)
9.0
Picha ya Profaili ya RockStone435
RockStone435

Hello - Nice to see your project posted here. I want to know if it will be an automated process linked to some cron job? Thanks.

$2000 USD kwa siku 30
(Maoni 676)
8.9
Picha ya Profaili ya michale21
michale21

Hi, Greetings from Votive Technologies!! We have good experience of Core PHP, PHP frameworks, PHP- CMS's, PHP - Ecommerce, Mysql, Responsive Web Designs, Ajax, jquery, JavaScript, Json, Photoshop, Graphics, Banne Zaidi

$3000 USD kwa siku 30
(Maoni 487)
9.1
Picha ya Profaili ya kabirchy
kabirchy

Hi there - My name is Khorshed. I've read your brief and can see that you’d like to build a website which will work with JSON API. I can help you get this done. I am very good with PHP and mysql and can develop what y Zaidi

$2352 USD kwa siku 25
(Maoni 1216)
9.4
Picha ya Profaili ya drupaltonic
drupaltonic

Hi, seems a great project to work on. We have gone through the brief and will be great to discuss the project and this proposal further with you. We are a web development company based in India. We do mostly Open Sourc Zaidi

$2941 USD kwa siku 30
(Maoni 127)
8.6
Picha ya Profaili ya aqualeafitsolpl
aqualeafitsolpl

A proposal has not yet been provided

$3000 USD kwa siku 30
(Maoni 165)
8.6
Picha ya Profaili ya bdsiddhi
bdsiddhi

Hi There, Greetings !! We have a team of 25 full time, dedicated developers having expertise in iPhone, Android, Joomla, Magento, Wordpress, Prestashop, OpenCart & Core PHP. We are willingly to work on this project Zaidi

$3940 USD kwa siku 30
(Maoni 254)
8.9
Picha ya Profaili ya taskmanageryd
taskmanageryd

A proposal has not yet been provided

$3000 USD kwa siku 25
(Maoni 314)
8.8
Picha ya Profaili ya fashionMobile7
fashionMobile7

A proposal has not yet been provided

$2647 USD kwa siku 30
(Maoni 18)
7.5
Picha ya Profaili ya latatestTech
latatestTech

let's chat for 100% quality

$2941 USD kwa siku 60
(Maoni 190)
8.2
Picha ya Profaili ya C0RETECHIES
C0RETECHIES

Hey hi I am Varun, we are experts in JSON. I can assure you that we can complete this project for you. Please initiate a personal message to discuss further.

$2500 USD kwa siku 10
(Maoni 112)
7.9
Picha ya Profaili ya monoLancer
monoLancer

Hi, I read all your requirements and understand what you need. I am very familiar with JSON API because I've made and used JSON API for my previous mobile app. Frankly I am very good in web service development and co Zaidi

$2647 USD kwa siku 30
(Maoni 43)
7.5
Picha ya Profaili ya alyaspk
alyaspk

Dear Employer, I am ready to work on your required task and can assure 100% satisfaction. You can read reviews/details about my past work on my profile http://www.freelancer.com/u/alyaspk.html Looking forward to Zaidi

$6000 USD kwa siku 30
(Maoni 194)
7.5
Picha ya Profaili ya ngocleanh
ngocleanh

this is my job. i can do it.

$2700 USD kwa siku 5
(Maoni 218)
7.5
Picha ya Profaili ya iglobalvn
iglobalvn

Hi, sir. I have read your requirement carefully. I am interested to work for this project and i will provide within or before 30 days . If you wanna confirm me, please check my work history and portfolio kindl Zaidi

$3000 USD kwa siku 30
(Maoni 98)
7.4
Picha ya Profaili ya awadeshm
awadeshm

Hi, my name is Awadesh. I am. Dr versed with API and neon and can do this task. Let's discuss more

$1500 USD kwa siku 12
(Maoni 71)
7.0
Tuma Mradi Kama huu