Develop Python script to handle large XML file using SAX parser and load to MySQL tables

Imekamilika Ilichapishwa Miaka 7 iliyopita Kulipwa wakati wa kujifungua
Imekamilika Kulipwa wakati wa kujifungua

We currently have a multi-step process which loads XML data into a MySQL database on a weekly basis. A file size increase of the incoming XML files necessitates a switch to a SAX parser from the current DOM parser.

Current state:

- Existing Python script uses [login to view URL] DOM and loops through 15 weekly XML files, performing a minor transformation to each using an XSLT file, and writes transformed XML to temporary .xml files

- The temporary .xml files are loaded to seven MySQL tables using LOAD XML LOCAL INFILE

Requirements:

- Develop a Python script which replicates the above steps into a single script that uses a SAX parser which can handle XML files with size of at least 2GB

- The script should parse through each of the XML files and load the data to the target AWS MySQL tables

- Script should unescape html elements (&) which exsit in XML prior to writing to database tables (see: [login to view URL])

- Script must process files in batches (7, 7, 1) so that DB scripts can be executed between each batch

- Code should be fully commented

- If applicable, please include instructions for installation of any required modules, etc.

I will provide the following:

- sample XML file

- existing Python script

- XSLT file currently used to transform XML

- SQL which is currently used to load the MySQL tables

Please contact me for more information.

MySQL Python SQL XML

Kitambulisho cha Mradi: #12855450

Kuhusu mradi

15 mapendekezo Mradi wa mbali Ipo mtandaoni %project.latestActivity_relativeTime|badilisha%

Imetuzwa kwa:

lkhelladi

Hello, I'd be glad to implement the Python script for XML parsing. Looking forward to chat with you soon for more details. Best regards,

$177 USD kwa siku 2
(Maoni 170)
7.4

15 wafanyakazi huru wanazabuni wastani wa $193 kwa kazi hii

gangabass

I'm expert in data processing with hundreds of excellent reviews here that's why I'm sure you'll be impressed with my work. I can create the script you want but I need to clarify one thing: are you talking about wri Zaidi

$200 USD kwa siku 2
(Maoni 401)
7.5
eagleblackdesign

Hello! We are a professional team of web developers with huge experience in using python for custom webapps based on django and odoo. Please provide more details about your project. We are available and will be happy Zaidi

$250 USD kwa siku 7
(Maoni 23)
6.5
ykStudios

Hi there, I'm highly experienced in python/web services. I can build a more optimized xml parsing script for large files, within about a week. The exact solution will depend on how the existing service is deploye Zaidi

$350 USD kwa siku 7
(Maoni 8)
4.7
yh2013

I have the required expertise for developing your project. Check my developed projects and reviews of my portfolio.

$111 USD kwa siku 7
(Maoni 8)
4.0
kaleshwarchand83

Hi, I have worked with python, and with large xml and csv files (upto 10GB in size) for a SAAS service that I was working on. I am sure I can get python to handle your xml files using SAX parser and get it imported Zaidi

$30 USD kwa siku 3
(Maoni 16)
3.8
imanolie

Hi there, I am an experienced developer and I can do this for you. Also to investigate a bit more about more efficient ways of parsing data. Give me the details you were talking about to have a look. Lopking for Zaidi

$250 USD kwa siku 5
(Maoni 4)
3.1
jme57f6f75551333

Computer Engineer. More than 20 years of experience. Expert in, but not limited to, GNU/Linux, BSD, Computer Security. Will lead your project to success.

$166 USD kwa siku 3
(Maoni 0)
0.0
KSVWebster

SALUTATIONS How are you? I hope you are doing well and the season is going great for you and your business. I and my team have gone through your requirements for this project. And one thing I can assure you that t Zaidi

$150 USD kwa siku 3
(Maoni 0)
0.0
yogeshrasal10

Below will be my plan: 1. Understand existing script and working as a first milestone. 2. Start working on new requirement and create a basic working prototype. 3. Corner cases and exception handling as a next miles Zaidi

$166 USD kwa siku 4
(Maoni 0)
0.0