Gather Your Daily Digest with Huginn

If you find you have a bunch of different sites you want to read on a regular basis, it can be time consuming to go to each site.

Furthermore, you might miss something. Maybe your list of sites is long. Comics to read, latest news, and so-on. For this, I'd recommend Huginn. Huginn is an agent-based server that, when scheduled, will perform certain actions for you. Think of it as an open-source version of IFTTT or Zapier. With Huginn, we're going to assemble a list of updates and send them out via email daily (assuming there's events to send out, of course).

Comic Downloads

Let's start with the example that's provided when you first install Huginn: the xkcd comic downloader. This uses the Website Agent to download the main page of the xkcd site, then extract out the comic URL, the comic title, and the hover text.

In my case, I also needed to modify the URL. xkcd doesn't lead their image URLs with "https", so I added it using a template.

{
  "url": "https://xkcd.com",
  "mode": "on_change",
  "expected_update_period_in_days": 5,
  "extract": {
    "orig_url": {
      "css": "#comic img",
      "value": "@src",
      "hidden": "true"
    },
    "title": {
      "css": "#comic img",
      "value": "@alt"
    },
    "hovertext": {
      "css": "#comic img",
      "value": "@title"
    }
  },
  "template": {
    "url": "https:{{orig_url}}"
  }
}

You can use this as a guideline for pulling data from other comics online. You should familiarize yourself with CSS selectors so you can pull the required information out of the page.

Finally, there are some other settings you should change. Set the schedule to a reasonable time (I have mine set to an hour before the digest goes out), and make sure the Receiver (who gets the agent event) has the Comic Formatter in it. There is a control called expected_update_period_in_days which is used to perform a health check on the agent. Set this to an appropriate timeframe for your comic based on how often they update. xkcd is set to 5 by default, but some comics update daily and so a value of 1 should be used in those situations. Using this can help tell you, at a glance, if an agent isn't working (or the artist is taking a vacation).

Comic Formatting

The comic formatting agent (which is set up by default when you first install Huginn) is awesome on it's own. It creates a header, displays the image, and puts the hovertext underneath. I didn't feel the need to change this, although if you want a fancier looking email, you might want to fiddle with this.

Pulling Reddit Threads

An additional item I added to into my digest is the top music streaming post from /r/music. It gives me one song to listen to that I (hopefully) haven't heard before.

Reddit has a JSON API you can work with without an API key. Simply add .json to the end of the URL and you'll get an easily parseable object with information about the post. You can then customize your output however you want to feed it into the email. Just make sure what you want displayed is put into the "message" attribute on the resultant payload.

Feeding It All Into An Email

The Email Digest Agent provides you with everything you need already. All the base logic for handling the email digest already exists for you within the agent. Remember to set the Sources to what agents should be contained in the email. This isn't the comics themselves, but instead the Comic Formatter agent.

When Should I Send Out My Daily Digest?

This is really up to you. The reading can be nice in the morning when you first wake up, but can be disruptive if you have a lot of reading material and you like to get your work done early.

My digest sends out the email at 4pm, roughly when I am on my bus ride home from work. I get something to read to decompress from a busy day. On the weekends, it helps remind me that I should get ready for dinner as well. You could also trigger it using other events as well, if you so choose.

Alternatives to Huginn

Another agent-based system I'm aware of is called Beehive, although I've never personally used it myself.