PySpark program to generate large dataset with 100 000 columns and 50 million rows

$10-30 USD

Inaendelea

Imechapishwa

over 1 year ago

$10-30 USD

Kulipwa wakati wa kufikishwa

I need a PySpark program to generate a large dataset with 100 000 columns and 50 million rows. I should be able to set the number of dimension columns (i.e. columns with non-numeric values such as Country, State, Suburb, Product etc). The rest of the columns must be all numerically populated with random floating-point numbers. The program's output needs to save the data to a single parquet or CSV file. I need to be able to set up the dimension values with CSV tables in the format below. Dimension name: Country File name: [login to view URL] File contents: 1,United States 2,United Arab Emerates 3,Saudi Arabia Random numbers must be picked from the file above to populate the dimensions.

PySpark

Spark

Apache Spark

Kitambulisho cha mradi: 34964959

Kuhusu mradi

3 mapendekezo

Mradi wa mbali

Inatumika 2 yrs ago

Unatafuta kupata pesa?

Barua pepe

Faida za kutoa zabuni kwenye Freelancer

Weka bajeti yako na muda uliopangwa

Pata malipo kwa kazi yako

Eleza pendekezo lako

Ni bure kujiandikisha na kutoa zabuni kwa kazi

Imetolewa kwa:

@bizzarriandrea81

Hi, I'm a senior data scientist and Python developer with more than 8 years experience. As a data scientist, I collaborated with several machine learning engineers and data engineers to design, develop, and evaluate some ML models on large datasets. I have hands-on experience on big data analysis and data visualization. I also have strong cooperation and communication skills. PERSONAL SKILLS 1. Data Science with Python and R 2. Machine Learning and Statistical Analysis with Python and R 3. Data Visualization through Tableau and Grafana 4. Data Analysis with Python(Numpy, Pandas) 5. DBMS(MySQL, Postgresql, MongoDB, Hive, Cassandra) 6. Deep Learning & Computer Vision(OpenCV, Keras, Tensorflow, Pytorch) 7. Natural Language Processing 8. Big data analysis with PySpark "Satisfy the client with my ability and passion" This is my slogan here. I hope you will be interested in me. Thanks.

$30 USD ndani ya siku 2

5.0

(1 hakiki)

1.8

3 wafanyakazi huru wana zabuni kwa wastani $27 USD kwa kazi hii

@Valuesolutions

Hello, I hope this finds you well. I have just seen your project requiring; Spark PySpark Apache Spark I believe that my 10-year experience in this field is what you need right away. Avoid the headache of looking further. Let's save time and focus on the real task. My proposed timelines and budget are just placeholder and an open for negotiation to increase or decrease as per the full requirements at hand. Allow me to prove how better my review can be. Smile all the way! Click the message button, so initiate the conversation. Regards, Fridah

$20 USD ndani ya siku 7

0.0

(0 hakiki)

0.0

@albanusklo

Dear Client, Greetings and hoping you are doing well, i welcome you to my profile where quality and client satisfaction is the Priority. CERTIFIED EXPERT IN LISTED PROJECT SKILLS & REQUIREMENTS, After KEENLY reading your description and being in possession of all CLEARLY STATED REQUIRED SKILLS (Apache Spark, Spark and PySpark) as this is my area of PROFESSIONAL SPECIALISATION having the above QUALIFICATIONS, i hereby humbly request you to consider my bid for QUALITY and PROFESSIONAL work. ###// No timeline & Quality Excuses// Unlimited Revisions// 100% SATISFACTION GUARANTEED### Message Me We Discuss More About The Project. incase the project is deleted please hire me directly using the following link; https://www.freelancer.com/u/topgradeclubltd THANK-YOU & WELCOME.

$30 USD ndani ya siku 1