Reddit Scraper and Telegram Bot

Question

I’ve got an idea to scrape some scientific news from the "Science" subreddit and broadcast it via a telegram bot into my telegram channel. I’ve constructed these two simple code fragments in Python for each of these tasks. Now I’m wondering what is the best way to combine them in one solid block of code so that the bot can automatically send the info that has been scraped to the channel each time the program is executed. Both scripts work just fine individually. Please advise.

Reddit Scraper

import praw

# assigning Reddit API data
# see further instructions here --> https://www.reddit.com/prefs/apps
reddit = praw.Reddit(client_id='XXXX', \
                     client_secret='XXXXXXXXXXXXXXXXXXXXXXX', \
                     user_agent='science_bot', \
                     username='XXXXXX', \
                     password='XXXXXXXXXXXXXXXXXX')

# select a subreddit you want to use for scraping data
subreddit = reddit.subreddit('science')
new_subreddit = subreddit.new(limit=500)
print("\t", "Digest of the latest scientific news for today: \n")
for submission in subreddit.new(limit=5):
    print(submission.title)
    print(submission.url, "\n")

Posting Telegram Bot

import requests

def telegram_bot_sendtext(bot_message):
    
    bot_token = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
    bot_chatID = '@XXXXXX'
    send_text = 'https://api.telegram.org/bot' + bot_token + '/sendMessage?chat_id=' + bot_chatID + '&parse_mode=Markdown&text=' + bot_message

    response = requests.get(send_text)

    return response.json()
    

test = telegram_bot_sendtext("Testing my new Telegram bot.")
print(test)

Thanks in advance!

The fastest and ugliest solution is to copy/paste the contents of the two scripts into one. I’m not sure I understand what the issue is. — AMC
There are many solutions, you can put scripts into functions and then glue them with some main function together. Or you can move each script under its own module (consider using function/objects there as well) and glue them in 3rd script, which will import them and do the stuff. To be honest, it is hard to understand what exactly problem is, try to add more info about your issue, like what you tried, what didn't succeed etc. — maxwell
Hi guys, thank you for stepping in. Basically I'm looking for the simplest algorithm which will allow extracting results from the first block of code (Reddit Scraper) into the second one (Posting Telegram Bot) and the simplest ways of its implementation. For instance, I can turn the first part into a function, but what is the most straightforward way to integrate it into the second block of code? — web_tracer
fyi it's scrape (and scraper, scraped, scraping) not scrap — balmy

web_tracer web_tracer · Accepted Answer · 2020-03-06T20:37:15

I've solved the problem using the following code structure. Kudos to @maxwell for a simple and elegant idea.

import telegram
import telebot
import praw

bot_token = 'XXXXXXXXXXXXXXXXXXXXXXXXX'
bot_chatID = '@your_channel_name'
bot = telebot.TeleBot('XXXXXXXXXXXXXXXXXXXXXXXXX')

reddit = praw.Reddit(client_id='XXXXXXXXXXXXXX', \
                     client_secret='XXXXXXXXXXXXXXXXXXXXXXXX', \
                     user_agent='your_bot_name', \
                     username='your_reddit_username', \
                     password='XXXXXXXXXXXXXX')

def reddit_scraper(submission):
    news_data = []
    subreddit = reddit.subreddit('name_of_subreddit')
    new_subreddit = subreddit.new(limit=500)
    for submission in subreddit.new(limit=5):
        data = {}
        data['title'] = submission.title
        data['link'] = submission.url
        news_data.append(data)
    return news_data

def get_msg(news_data):
    msg = '\n\n\n'
    for news_item in news_data:
        title = news_item['title']
        link = news_item['link']
        msg += title+'\n[<a href="'+link+'">Read the full article --></a>]'
        msg += '\n\n'

    return msg

subreddit = reddit.subreddit('name_of_subreddit')
new_subreddit = subreddit.new(limit=500)
for submission in subreddit.new(limit=1):
    news_data = reddit_scraper(submission)
    if len(news_data) > 0:
        msg = get_msg(news_data)
        status = bot.send_message(chat_id='@your_channel_name', text=msg, parse_mode=telegram.ParseMode.HTML)        
        if status:            
            print(status)
else:
    print('No updates.')

Reddit Scraper and Telegram Bot

1 Answers