Find Jobs
Hire Freelancers

Node save web page and send to s3 with npm website-scraper and aws-sdk packages

$30-250 USD

Imekamilika
Imechapishwa over 5 years ago

$30-250 USD

Kulipwa wakati wa kufikishwa
Write a node script which scrapes a webpage using the npm package website-scraper [login to view URL] you must use your own AWS account, the script will use [login to view URL]() to set credentials Use default options and save just the one page / URL to a tmp folder Then use the AWS SDK ( [login to view URL] ) and send the folder to an s3 bucket. For each file set the correct content-type Here is some code to get you started var recursive = require("recursive-readdir"); const options = { urls: ["[login to view URL]"], directory: "/tmp/yahoo" }; scrape(options, (error, result) => { recursive([login to view URL], function(err, files) { for(i = 0; i< [login to view URL];i++){ file = files[i]; file = [login to view URL](1); var params = { Body: // set file content here Bucket: "bucket_name", Key: file, ContentType: // set content type here ACL: 'public-read', }; [login to view URL](params, function(err, data) { if (err) [login to view URL](err, [login to view URL]); // an error occurred else [login to view URL](data); // successful response }); } }); }); Budget is less than $100
Kitambulisho cha mradi: 18275015

Kuhusu mradi

3 mapendekezo
Mradi wa mbali
Inatumika 5 yrs ago

Unatafuta kupata pesa?

Faida za kutoa zabuni kwenye Freelancer

Weka bajeti yako na muda uliopangwa
Pata malipo kwa kazi yako
Eleza pendekezo lako
Ni bure kujiandikisha na kutoa zabuni kwa kazi
Imetolewa kwa:
Picha ya Mtumiaji
I am a hobbyist JavaScript developer experienced in NodeJS. I have also worked on a couple personal projects which have involved website scrapping. For this reason, I believe I would be a good fit for this project. For a examples of my past work, I would encourage you to take a look at my other personal projects on my github page [login to view URL] Please message me with further details about the project to see if I can help. Thanks for your consideration, Samuel
$94 USD ndani ya siku 5
5.0 (2 hakiki)
3.2
3.2
3 wafanyakazi huru wana zabuni kwa wastani $148 USD kwa kazi hii
Picha ya Mtumiaji
Hi there, I am Node.js AWS expert, I can scrap the the website with your required modules and can save it to S3 bucket and can give you demo on my machine before delivering the code base. I have done several projects with AWS serivces like S3, Lambda, Kinesis, SQS,SES, SNS, EC2 etc. you may have look at my profile. I have done several projects in Node.js with AWS services. I have 4+ years of experience in software development. I am Sr. Full Stack Developer having experience in Node.js, REST APIs, JavaScript, AngularJS(1/2/4/5),ExpressJS, Google Cloud Function,Google Firebase function, AWS ,SQL(MySQL, PostgreSQL, MS SQL Server, Sqlite3), NoSQL(MongoDB, CouchDB), Linux, Git,Nginx. I am good to deliver your code base in $100 Looking forward to work with you Thanks...
$111 USD ndani ya siku 1
5.0 (29 hakiki)
4.7
4.7
Picha ya Mtumiaji
I have worked with node scraped pages using python So it will be an opportunity for me to save webpages via node. I have my AWS account and I am Cloud Engineer by profession ensuring continuous Integration and Delivery of services using AWS and automation tools like Chef, Jenkins, Docker, Kubernetes Specialities: Continuous Integration, Continuous Delivery, Docker Containers, Container orchestration platforms, Infrastructure Automation, Microservices, Monitoring and log management List of Software, Tools used/working on • Automation - Chef, Vagrant, Docker, Kubernetes • CI/CD - Jenkins • Amazon Web Services CloudWatch, S3, EC2, Load Balancers, Auto Scaling, VPC, VPC Peering, VPC - Site VPN, IAM, Cloud Front, Elastic Search, RDS, SES, SQS, SNS, ElastiCache, RDS, EC2 Container Service
$240 USD ndani ya siku 3
5.0 (2 hakiki)
2.2
2.2

Kuhusu mteja

Bedera ya UNITED STATES
Mount Sinai, United States
5.0
32
Njia ya malipo imethibitishwa
Mwanachama tangu Jul 30, 2010

Uthibitishaji wa Mteja

Asante! Tumekutumia kiungo cha kudai mkopo wako bila malipo kwa barua pepe.
Hitilafu fulani imetokea wakati wa kutuma barua pepe yako. Tafadhali jaribu tena.
Watumiaji Waliosajiliwa Jumla ya Kazi Zilizochapishwa
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Onyesho la kukagua linapakia
Ruhusa imetolewa kwa Uwekaji wa Kijiografia.
Muda wako wa kuingia umeisha na umetoka nje. Tafadhali ingia tena.