Problem: Trying to pull tweets via tweepy using Cursor. I want to make sure I don't pull tweets I previously pulled.
Here is working code:
import tweepy
import pandas as pd
import numpy as np
ACCESS_TOKEN = ""
ACCESS_TOKEN_SECRET = ""
CONSUMER_KEY = ""
CONSUMER_SECRET = ""
# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
# Creation of the actual interface, using authentication
api = tweepy.API(auth, wait_on_rate_limit=True)
csvFile = open(r'filename', 'a')
#Use csv writer
headers = ['UserName', 'Tweet', 'TweetId', 'tweet_date', 'source', 'fav_count', 'retweet_count', 'coordinates', 'geo']
# definitions for writing to CSV
csvWriter = csv.writer(csvFile, lineterminator='\n')
# write the headers once
csvWriter.writerow(headers)
handles = ['pycon', 'gvanrossum']
previousTweets =
['222288832031240000',
'222287080586362000',
'222277240178741000',
'221414283844653000',
'221188011906445000',
'205274818877210000']
for handle in handles:
for status in tweepy.Cursor(api.user_timeline, screen_name= handle, tweet_mode="extended").items():
if status.id not in previousTweets:
csvWriter.writerow([status.user.name.encode('utf-8'), status.full_text.encode('utf-8'), status.id, status.created_at, status.source,
status.favorite_count, status.retweet_count, status.coordinates, status.geo])
print(handle)
This takes a long time and becomes unusable if you want to have a PreviousTweet list of over 75 tweets. Does anyone know a better way to filter out old tweets when using Tweepy and the Cursor function?
from Tweepy: Ignore previous tweets to improve optimization
No comments:
Post a Comment