In this video we will look at web scraping using Python and the BeautifulSoup library. This is an introductory level tutorial. All beginners welcome
Final Code Gist:
| import requests | |
| from bs4 import BeautifulSoup | |
| from csv import writer | |
| response = requests.get('http://codedemos.com/sampleblog/') | |
| soup = BeautifulSoup(response.text, 'html.parser') | |
| posts = soup.find_all(class_='post-preview') | |
| with open('posts.csv', 'w') as csv_file: | |
| csv_writer = writer(csv_file) | |
| headers = ['Title', 'Link', 'Date'] | |
| csv_writer.writerow(headers) | |
| for post in posts: | |
| title = post.find(class_='post-title').get_text().replace('\n', '') | |
| link = post.find('a')['href'] | |
| date = post.select('.post-date')[0].get_text() | |
| csv_writer.writerow([title, link, date]) |
Become a Patron: Show support & get perks!
http://www.patreon.com/traversymedia
Website & Udemy Courses
http://www.traversymedia.com
Follow Traversy Media:
http://www.facebook.com/traversymedia
Tweets by traversymedia
http://www.instagram.com/traversymedia
Source
