I'll provide more info on successful shortlisting of Freelancers (see criteria below) but essentially what I need you to do is:
1. Create a new Database using the Data Schema I will provide (2 tables - products table with 42 columns/data points, deals table with 63 columns/data points approximately) on my server.
2. Create scripts (PHP or otherwise) that can do the following, in this order:
- Download a compressed file from the URLs I provide (there are approximately 15 URLs)
- Unzip the file which will contain a large CSV file (one file could have around 1 million rows of data).
- Extract the data in the CSV file and import it into the Database you created in the first action.
- Sends an email to me with a volume of the products and the deals imported
- There must be a script per URL so I have control of which scripts I run (if I want to just import the data from one URL manually I can do)
- A Cron job will need to be created so I can configure when the scripts run automatically on my server each day
- I need a very simple front end HTML page that requires a login to access with a summary of the imports (I will provide flat HTML files for you to use, you'll just need to integrate it with your scripts) and the ability to trigger the scripts.
- The server I have will have (at best) 2GB RAM and 2 processors. The scripts need to run optimally and complete quickly (I will allow a maximum of 1 hour for the largest script / import to run), I cannot approve this project as complete unless this happens so please think about how you plan to build this with the potential size of the CSV files..
- The Products Import script must run a check to see if a Product exists already in the Database (using unique product id) before importing. If it exists already in the DB then do not import the product.
- The Deals import script must clear the DB first before importing new deals to ensure they are the most recent deals. You can use the Retailer ID to clear Deals from specific URLs if required.
- The CSV files are structured in exactly the same way, so the mappings only need to be created once as they can be applied to all the CSV files that will be downloaded
- There are other options when it comes to the files to be downloaded, if you think this would be better
File format and compression options available of the files to be downloaded:
- XML (no DTD)
- XML (DTD 1.5)
- XML (DTD 1.4)
- Ill review all submissions and bids
- Ill shortlist the freelancers that I think can do the work based on their bid and what they write in their bid. Automated bidding will be ignored and you'll be removed from the project.
- Shortlisted Freelancers will get a message with the Data Schema attached and an example of one of the CSVs with an opportunity to ask any questions.
- Ill choose a Freelancer to work on the project.
To add: Im happy for you to use a different language other than PHP, for example Node,js.
36 freelancers are bidding on average £190 for this job