The above definition is rather brief, and leaves a lot of room for imagination. But I’m not going to elaborate right now, and instead, I’ll just present a crude prototype.
This is fairly technical. So Linux/BSD/OS X hackers, read on!
(Windows users: please leave a comment below so that I can be motivated to make this work for you too!)
The setup is very simple: pandoc + make + darcs. Let me explain each of them one by one:
Pandoc is a tool (and software library) translating documents from simple markup formats to multiple output formats like HTML, Latex, PDF, etc. Writing in a format closer to pure text (e.g., Markdown) is a thousand times more productive than in a rigid format like HTML.
Make automates tasks such as batch building HTML out of text, syncing to the website, etc.
Darcs (or any other version control system) tracks document versions, lets you roll back changes, work from multiple locations, collaborates with others, and is mandatory in any modern document systems. But we’ll make it optional here.
The work flow is also simple: edit (in your favorite editor), make (using the make tool), view (in any browser), with occasional book keeping activities such as adding a new document to the
Makefile (explained below), or recording changes using
First of all, we need to ensure the
make command is available. Make comes with most systems as a developer package.
make has been properly installed, open a terminal window and type:
It will report its version number, or otherwise a command not found error.
Next, we need to get Darcs. This step is optional if you don’t want version control. You can download the latest binary version from the Darcs website for your platform, unzipping and putting it in
/usr/local/bin/ or some other place in
darcs has been properly installed, in a terminal window, type:
It should report its version, or otherwise a command not found error.
darcs, we can get the latest sitetool software:
darcs get http://www.thev.net/sitetool
It will create a new directory called
sitetool, and download everything to it. Because this package also includes pre-compiled binaries for various platforms, and
darcs usually will not report any message during the download, please be patient while you are waiting for
darcs to finish.
Alternatively if the above fails or you don’t have
darcs, you can get the software from http://www.thev.net/download/sitetool.zip, and unpack it to any place in your local hard disk.
After we get the software, we have two ways to prepare it:
Use pre-compiled binary
In a terminal window, go to the
sitetool directory and type:
It will automatically detect your platform, and choose the right pre-compiled binary of the
bin/markdown2html tool. Only Linux i386 and OS X (Intel) 10.5 or 10.6 are supported at the moment. If your platform is not supported, it will report an error.
Compile from source
If you want to be a developer, or if the binary installation fails, you can always compile from source. Since the software is written in the Haskell language, We shall first install the Haskell Platform before we proceed. The Haskell Platform installs the Glasgow Haskell Compiler (GHC), as well as a package manager called Cabal.
cabal command is installed in
~/.cabal/bin/ directory, and if it’s not already in your
PATH, we can add it:
This setting can be made permanent if you put it in the file
~/.profile (depending on your shell setting).
cabal command, we install Pandoc:
cabal update cabal install pandoc
cabal update updates a local package cache from the Hackage website, and the second command downloads the latest Pandoc package, compiles it with GHC, and installs the library. In this process, it also downloads and installs required libraries due to dependencies.
Then in the
sitetool directory, we can type:
to compile the
bin/markdown2html tool. If Pandoc has been successfully installed, the compilation process should also finish successfully. Report by posting a comment to this page if you encounter any problem.
Optionally, we may compile Darcs from source too:
cabal install darcs --flags="http -curl"
The above command installs
darcs to the same
sitetool itself can be renamed, or moved to any place you like. But all your future documents must reside in this directory, and you need to change to this directory before performing any operation in the command terminal.
To verify the installation is successful, we can build a sample document that comes with the package by:
This will produce a HTML file
sample.html, which can then be loaded into a browser window for viewing.
To keep the software up to date, just use the command:
darcs pull http://www.thev.net/sitetool
This will pull any new changes from the repository, and update your local copy automatically. The do one of the following:
make binaryto install the latest binary.
make allto re-compile.
In some very unlikely events, conflicts may arise when
darcs is pulling from the remote darcs repository. Please learn more about Darcs and try to resolve the conflicts yourself!
To understand the work flow, we shall view the
Makefile, part of which may look like this:
UNAME:=$(shell uname -s) GHC=ghc RM=rm -f ROOT=. BINDIR=$(ROOT)/bin PUBDIR=$(ROOT)/public MARKDOWN2HTML=$(BINDIR)/markdown2html CSS=--template-option=css=style.css EXTRA= OPTIONS=$(CSS) --outputdir=. $(EXTRA) PUBOPTIONS= PUBS= DOCS=$(PUBS) sample.html ... docs: $(DOCS) ... %.html: $(ROOT)/%.txt $(MARKDOWN2HTML) $(OPTIONS) $< ...
Here, the variable
DOCS contains the list of documents we’ll be building, and by typing command:
we build all documents from their source form (files with extension
.txt) into HTML (files with extension
.html). For example, if we have in the
DOCS=index.html todo.html backup-plan.html
Then these HTML targets will be built from three source files
index.txt todo.txt and
backup-plan.txt. If a source file doesn’t exist,
make will report an error.
The source document can be written in either plain text, or Markdown syntax. The
sample.txt file already gives an example of the latter. We are actually using the Pandoc extension of Markdown, which brings additional features such as title block, syntax highlighting, HTML template, etc.
markdown2html tool does one more thing: shortcut for cross-linking among local documents. The usual Markdown syntax allows links in the form of
[text](url), where the
url has to be fully spelled out. Here we allow the
[text](name) syntax, where
name is a local filename without extension.
For example, if we write
[link back to the index page](index), it is translated to this: link back to the index page.
The biggest benefit of this tool is that we can use our familiar text editors to write documents that have rich formatting, cross links, and are directly viewable in a web browser. It’s very close to a personal Wiki system, but without any fancy web sever, or the drawbacks of using the lame edit box in a browser.
Lastly, we can maintain a full history of all documents using
darcs, as well as replicate them to multiple machines to support a distributed working style. See Darcs website for more information on how to use it.
For advanced users, here are some advanced features.
style.css is a variation of the Typogridphy CSS wizardry, which gives the same clean and minimalistic look at what you see on this site. You can change it to anything you like, or even add more than one CSS files by modifying the definition of
CSS in the
Makefile. For example:
This will enable two CSS files in the generated HTMLs:
HTML templates are supported since Pandoc 1.4. Indeed, the web page you see on this site are completed with TOC sidebar, footer, and disqus comment systems. This is achieved by using a custom template, which can be enabled by changing the
OPTIONS setting in the
This will make all generated HTMLs from a template file called
html.template in the directory where you run
make. Template variables can also be set by adding
--template-option="var=value" to the
OPTIONS setting. If you have installed Pandoc 1.4, a sample template can be found at
~/.cabal/share/pandoc-1.4/templates/html.template. More details on how to use templates are given in the Pandoc User’s Guide.
You may copy any generated HTML file manually to a website to publish, or you can use Make to automate this process. You may have already noticed that in the
Makefile we have a
PUBS variable. Just put all the files you want to make public in the
PUBS variable, and the rest in the
DOCS variable after
$(PUBS). For example, part of our
Makefile may look like this:
PUBOPTIONS=--template-option="author=Some Author" PUBS=index.html album.html DOCS=$(PUBS) diary.html
The above configuration defines in total three documents,
diary.html, where the first two are classified as public. Regardless of public or private, all HTML files are viewable from the local folder.
Also note that in the above example we have
PUBOPTIONS setting the Pandoc template variable
Some Author, which is a means to provide extra settings for public documents when we publish them to websites. Doing just
make will only prepare all documents locally without using
PUBOPTIONS. Public documents are prepared differently by using:
This will generate all public HTMLs using the extra
PUBOPTIONS (in addition to existing
OPTIONS), and put them in a subfolder called
public, ready to be uploaded to websites or copied to Blogs. Private documents are not affected during this make.
To further automate the entire publishing process, we can put commands to the
public target in the
Makefile, such as uploading files from the
public subfolder to a FTP, or sync to a WebDAV folder, or things like that. Details are not discussed here.
tell application "TextWrangler" to set theFile to file of document 1 set htmlFile to do shell script "echo " & theFile & "|sed -e s/.txt$/.html/" set filePath to POSIX path of theFile set htmlPath to POSIX path of htmlFile set dirName to do shell script "dirname " & filePath set markdown2html to dirName & "/bin/markdown2html " do shell script "rm -f " & htmlPath do shell script "make -C " & dirName do shell script "test -f " & htmlPath & " || " & markdown2html & filePath tell application "Finder" to open htmlFile
After this, we’ll be able to see our script appearing as a menu item in TextWrangler’s Script menu. Clicking it will automatically execute the
make command in the folder of the current document, and then view the result HTML file in a web browser, which is usually Safari.
Even better, we can assign it a keyboard shortcut:
Note: The above script assumes the document you are editing is saved with a
.txt extension, and is already added to the
DOCS variable in the
Makefile. Otherwise it’ll fall back to a generic translation directly using
markdown2html instead of
make. The result will be just a basic HTML, and you’ll loose customizations in the
Makefile for things like CSS, and so on.
The combination of pandoc + make + darcs is very flexible, but still too technical to be used by a non tech-savvy user. I plan to evolve it into a single tool, with integrations to popular editors and browsers.
A more immediate goal is to support multiple users and site syncing so that this site can invite more authors.