Emacs and my Devlog
An awesome meetup with Emacs Org-Mode

My introduction to Emacs

I began using Emacs in between Vim since many months now. I love Vim for its fast Editing keybinds. Its my favorite in that regard. However, when I'm writing from scratch, Emacs feels a lot nicer.

I love the fact that it can write and edit text without moving between different modes. I have some Emacs keybinds like C-a, C-e, M-b, M-e mapped in my Vim too. These works as expected in Insert Mode and Command Mode. So I use Emacs and Vim both, whichever I feel like at the moment.

Now recently, I discovered something which made me rethink my Devlogging system. And I'm here three days afterward.

Emacs Org Mode

For people who are familiar with Markdown, Org mode can be thought of as a similar format.

Its a simple formatting system that allows basic text styling like bold, italic, underlined etc. Links, code blocks, and Block Quotes are supported too. No big-deal here.

There are certain quirks to remember when coming from Markdown based system. For example,

  1. Code Blocks are written withing #+begin_src <LANG> and #+end_src instead of Markdown's ``` <LANG> and ```.
  2. Block Quotes are written similarly within #+begin_quote and #+end_quote
  3. Links are written like [[LINK_URL][LINK_DESC]] instead of [LINK_DESC](LINK_URL)]
  4. Similarly, the markers for inline code, bold, italic are =, *, and / respectively

But, the big-difference comes in the tooling that is brought with Org-mode. Its available by default in Emacs (I'm using GNU Emacs), and it brings a plethora of features as is the like of Emacs packages.

Quality of Life improvements

Ofcourse, if we want, we can hide the control characters to make the org files a bit more readable.

(setq-default org-hide-emphasis-markers nil)
(setq-default org-hide-leading-stars nil)
(setq-default org-hide-macro-markers nil)
(setq-default org-hide-block-startup nil)
(setq-default org-hide-drawer-startup nil)

Multiple Export Options

Emacs's comes with multiple publishing options for the notes you make with Org-mode. It can export to, (among others)

  1. HTML
  2. LaTeX
  3. PDF

Now, we can do the same with Markdown as well. There are many programs available that can convert from Markdown to any of the aforementioned formats. Seriously, how can one forget Pandoc, the swiss army knife of document conversions.

But if someone's already using Emacs, then Org-mode is a good contender. And if we later change our mind, Pandoc supports Org-mode too, so we can convert our notes back to literally whatever format we want!

Endless Configurability

Since, the whole of Org mode is written in Elisp ofcourse, we can modify parts of it, or extend it any way. Assuming we can work with Elisp.

Publishing as HTML

The configuration we need to do at minimum to achieve org to html export is shown below, assuming our project is at ~/org and we want our HTML files to be generated in ~/org/public directory.

(setq org-publish-project-alist
      '(
        ("project-org"
         :base-directory "~/org"
         :base-extension "org"
         :publishing-directory "~/org/public"
         :recursive t
         :publishing-function org-html-publish-to-html)))

Most of the fields are self-explanatory. The publishing-function org-html-publish-to-html ensures, the org files are converted to HTML.

After this code piece is run, we can now publish our org files from ~/org with the following command,

(org-publish "project-org" t)

My Devlog site

I dived in and setup a project so that I can publish my whole notes into a HTML Website, my Devlog.

Let's organise a common structure for my org-file directory ~/docs/rxrj which will then get replicated in my published site.

Here is the structure I have on my system

  1. ~/docs/rxrj will contain the site files, think index, about, and possibly contact us
  2. ~/docs/rxrj/devlog will contain my devlog files. This is where we would put my devlog articles / posts.

Now, we want a Devlog section which will contain links to all files in my devlog sub-directory in a sorted manner. Newer posts show up on top of the list. We can manually write, into index file (~/docs/rxrj/index.org), link to every posts we write on my devlog, but why not make that automatic.

Here's the deal now, we write my index.org file normally. With whatever we want to add as we feel like. Links to About, Contact Us etc. The trick is we keep aside a section for Devlog in that page, and program my Elisp code to fill that section with the links to all the posts in my ~/docs/rxrj/devlog subdirectory sorted by the Date we wrote them.

First, we define a function that can generate a list of all the .org files in my ~/docs/rxrj/devlog subdir. Each entry in the list would contain the files' Title, Date, and Path. Also, this function sorts the devlog articles in reverse-chronological order, so newer articles show up first in the list.

(defun rxrj-get-org-entries (org-project-dir org-file-dir)
  (interactive)
  (let* ((files (directory-files org-file-dir t "\\.org$"))
       (entries '()))
    (dolist (file files)
      (with-temp-buffer
        (insert-file-contents file)
        (org-mode)
        (let
            ((title (cadr (assoc "TITLE" (org-collect-keywords '("TITLE")))))
             (date (cadr (assoc "DATE" (org-collect-keywords '("DATE"))))))
          (push
           (list :date date :title title :path (file-relative-name file org-project-dir))
           entries))))
    (sort entries
          (lambda (a b)
            (string>
             (plist-get a :date)
             (plist-get b :date))))))  

Now we define a function which can format these entries as Org Links

(defun rxrj-format-org-entries-as-links (entries)
(mapcar
 (lambda (entry)
     (format "- %s [[file:%s][%s]]"
             (plist-get entry :date)
             (plist-get entry :path)
             (plist-get entry :title)))
 entries))

The links in my index file would look like - [Date] [LINK]

Now we combine the two above in another function which modifies my index.org page by adding the links into the Devlog section

(defun rxrj-generate-subdir-index (project-dir subdir index-section-identifier)
(interactive)
(let* ((entry-links
          (rxrj-format-org-entries-as-links
           (rxrj-get-org-entries project-dir subdir)))
         (index-file (expand-file-name "index.org" project-dir)))
  (with-temp-buffer
      (org-mode)
    (insert-file-contents index-file)
    (goto-char (point-min))
    (re-search-forward index-section-identifier)
    (let ((beg (point))
            (end (save-excursion (org-end-of-subtree t t) (point))))
        (delete-region beg end))
    (insert (mapconcat #'identity entry-links "\n"))
    (write-file index-file))))

This will generate the entries for the provided subdir inside provided project-dir, format them as links using the two functions defined above. After that, it will open the index.org file in the project-dir, search for the section using index-section-identifier, and replace that section's contents with the formatted links.

And we can call this function like

(rxrj-generate-subdir-index "~/docs/rxrj" "~/docs/rxrj/devlog" "^\\* Devlog\n")

Static Files

Now for my static files (css, jpeg, png etc) which are in their own directory ~/docs/rxrj/static, here's how I added another item to the org-publish-project-alist to handle that.

("rxrj-static"
 :base-directory "~/docs/rxrj/static"
 :base-extension "css\\|\\|png\\|jpg\\|jpeg\\|gif"
 :publishing-directory "~/docs/rxrj/public/static"
 :recursive t
 :publishing-function org-publish-attachment)

In this case, we don't want any kind of processing done to the static assets, so a basic copying function org-publish-attachment is provided to publishing-function.

RSS Feed

A Devlog site would be incomplete without it incorporating the RSS feed url. Most readers won't like to visit sites to get articles. Email based delivery is the worst, as it requires reader to provide their email address.

But RSS Feed on the other hand is awesome. Readers user their Feed Reader to hit the site's RSS URL, and gets all of the updates it is interested in. Only the provider (Devlog site) has its url public, not the Reader's email.

I used a package called org-publish-rss to generate RSS feeds of my site. Since I want the feed to contain updates of only the articles I post (fils inside my devlog dir), i updated the project structure yet again to separate out devlog pages, and root directory pages.

The configuration for publishing the devlog pages would also contain the RSS feed generation config. It looks like

("rxrj-devlog"
 :base-directory "~/docs/rxrj/devlog"
 :base-extension "org"
 :publishing-directory "~/docs/rxrj/public/devlog"
 :recursive t
 :publishing-function org-html-publish-to-html
 :headline-levels 4
 :auto-preamble f

 :auto-rss t
 :rss-title "rxrj"
 :rss-link "https://riturajb.dev"
 :rss-description "Devlog of Rituraj Borpujari"
 :rss-with-content all
 :completion-function org-publish-rss)

The :completion-function org-publish-rss part says - after the publishing-function is run (org-html-publish-to-html), the next function to run is org-publish-rss, which takes care of generating the rss.xml file.

Other properties are self-explanatory.

Also, I would need another project config to copy the rss.xml file, which would be similar to how static files are handled.

This would look like,

("rxrj-rss"
 :base-directory "~/docs/rxrj/devlog"
 :base-extension "xml"
 :publishing-directory "~/docs/rxrj/public"
 :include ("rss.xml")
 :publishing-function org-publish-attachment)

All together now

Now to stitch the whole thing together here's what I need to do

  1. Generate Index entries for devlog subdir
  2. Modify index file to add the generated entries in Devlog section
  3. Use org-publish to publish devlog, static files, and rss.xml file

For Step #3, that I added another project config, rxrj which has just one purpose - contain the other project configs.

("rxrj"
 :components (
              "rxrj-devlog"
              "rxrj-index"
              "rxrj-static"
              "rxrj-rss"
              "rxrj-cname"))

For Step #1, and Step #2 we already have a combined function rxrj-generate-subdir-index.

Stitching the whole thing together, we would now have

(defun rxrj-publish ()
  (interactive)
  (rxrj-generate-subdir-index "~/docs/rxrj" "~/docs/rxrj/devlog" "^\\* Devlog\n")
  (org-publish "rxrj" t))

This single function does the whole steps and publishes the site, containing all the articles and the generated index page.

The whole code can be viewed in here, in my Github repo.

Future Improvements

I would, in near future, want to make the RSS feed generation a bit more controlled, so that it generates only a fixed number of articles in the file.

Similarly the index file should also contain only a fixed number of devlog links. The whole list should be hidden away in another page.

But I don't have much pages now. For now, I have the Site generator ready to use. Powered by Emacs, with Org-mode