The above definition is rather brief, and leaves a lot of room for imagination. But I’m not going to elaborate right now, and instead, I’ll just present a crude prototype.
This is fairly technical. So Linux/BSD/OS X hackers, read on!
(Windows users: please leave a comment below so that I can be motivated to make this work for you too!)
The setup is very simple: pandoc + make + darcs. Let me explain each of them one by one:
Pandoc is a tool (and software library) translating documents from simple markup formats to multiple output formats like HTML, Latex, PDF, etc. Writing in a format closer to pure text (e.g., Markdown) is a thousand times more productive than in a rigid format like HTML.
Make automates tasks such as batch building HTML out of text, syncing to the website, etc.
Darcs (or any other version control system) tracks document versions, lets you roll back changes, work from multiple locations, collaborates with others, and is mandatory in any modern document systems. But we’ll make it optional here.
The work flow is also simple: edit (in your favorite editor), make (using the make tool), view (in any browser), with occasional book keeping activities such as adding a new document to the Makefile (explained below), or recording changes using darcs.
First of all, we need to ensure the make command is available. Make comes with most systems as a developer package.
To verify make has been properly installed, open a terminal window and type:
make --versionIt will report its version number, or otherwise a command not found error.
Next, we need to get Darcs. This step is optional if you don’t want version control. You can download the latest binary version from the Darcs website for your platform, unzipping and putting it in /usr/local/bin/ or some other place in PATH.
To verify darcs has been properly installed, in a terminal window, type:
darcs --versionIt should report its version, or otherwise a command not found error.
Now with darcs, we can get the latest sitetool software:
darcs get http://www.thev.net/sitetoolIt will create a new directory called sitetool, and download everything to it. Because this package also includes pre-compiled binaries for various platforms, and darcs usually will not report any message during the download, please be patient while you are waiting for darcs to finish.
Alternatively if the above fails or you don’t have darcs, you can get the software from http://www.thev.net/download/sitetool.zip, and unpack it to any place in your local hard disk.
After we get the software, we have two ways to prepare it:
Use pre-compiled binary
In a terminal window, go to the sitetool directory and type:
make binary It will automatically detect your platform, and choose the right pre-compiled binary of the bin/markdown2html tool. Only Linux i386 and OS X (Intel) 10.5 or 10.6 are supported at the moment. If your platform is not supported, it will report an error.
Compile from source
If you want to be a developer, or if the binary installation fails, you can always compile from source. Since the software is written in the Haskell language, We shall first install the Haskell Platform before we proceed. The Haskell Platform installs the Glasgow Haskell Compiler (GHC), as well as a package manager called Cabal.
Usually the cabal command is installed in ~/.cabal/bin/ directory, and if it’s not already in your PATH, we can add it:
export PATH=~/.cabal/bin:$PATHThis setting can be made permanent if you put it in the file ~/.bashrc or ~/.profile (depending on your shell setting).
With the cabal command, we install Pandoc:
cabal update
cabal install pandocThe command cabal update updates a local package cache from the Hackage website, and the second command downloads the latest Pandoc package, compiles it with GHC, and installs the library. In this process, it also downloads and installs required libraries due to dependencies.
Then in the sitetool directory, we can type:
make allto compile the bin/markdown2html tool. If Pandoc has been successfully installed, the compilation process should also finish successfully. Report by posting a comment to this page if you encounter any problem.
Optionally, we may compile Darcs from source too:
cabal install darcs --flags="http -curl"The above command installs darcs to the same ~/.cabal/bin/ directory.
The directory sitetool itself can be renamed, or moved to any place you like. But all your future documents must reside in this directory, and you need to change to this directory before performing any operation in the command terminal.
To verify the installation is successful, we can build a sample document that comes with the package by:
make This will produce a HTML file sample.html, which can then be loaded into a browser window for viewing.
To keep the software up to date, just use the command:
darcs pull http://www.thev.net/sitetoolThis will pull any new changes from the repository, and update your local copy automatically. The do one of the following:
make binary to install the latest binary.make all to re-compile.In some very unlikely events, conflicts may arise when darcs is pulling from the remote darcs repository. Please learn more about Darcs and try to resolve the conflicts yourself!
To understand the work flow, we shall view the Makefile, part of which may look like this:
UNAME:=$(shell uname -s)
GHC=ghc
RM=rm -f
ROOT=.
BINDIR=$(ROOT)/bin
PUBDIR=$(ROOT)/public
MARKDOWN2HTML=$(BINDIR)/markdown2html
CSS=--template-option=css=style.css
EXTRA=
OPTIONS=$(CSS) --outputdir=. $(EXTRA)
PUBOPTIONS=
PUBS=
DOCS=$(PUBS) sample.html
...
docs: $(DOCS)
...
%.html: $(ROOT)/%.txt
$(MARKDOWN2HTML) $(OPTIONS) $<
...Here, the variable DOCS contains the list of documents we’ll be building, and by typing command:
makewe build all documents from their source form (files with extension .txt) into HTML (files with extension .html). For example, if we have in the Makefile:
DOCS=index.html todo.html backup-plan.htmlThen these HTML targets will be built from three source files index.txt todo.txt and backup-plan.txt. If a source file doesn’t exist, make will report an error.
The source document can be written in either plain text, or Markdown syntax. The sample.txt file already gives an example of the latter. We are actually using the Pandoc extension of Markdown, which brings additional features such as title block, syntax highlighting, HTML template, etc.
Our markdown2html tool does one more thing: shortcut for cross-linking among local documents. The usual Markdown syntax allows links in the form of [text](url), where the url has to be fully spelled out. Here we allow the [text](name) syntax, where name is a local filename without extension.
For example, if we write [link back to the index page](index), it is translated to this: link back to the index page.
The biggest benefit of this tool is that we can use our familiar text editors to write documents that have rich formatting, cross links, and are directly viewable in a web browser. It’s very close to a personal Wiki system, but without any fancy web sever, or the drawbacks of using the lame edit box in a browser.
Lastly, we can maintain a full history of all documents using darcs, as well as replicate them to multiple machines to support a distributed working style. See Darcs website for more information on how to use it.
For advanced users, here are some advanced features.
The default style.css is a variation of the Typogridphy CSS wizardry, which gives the same clean and minimalistic look at what you see on this site. You can change it to anything you like, or even add more than one CSS files by modifying the definition of CSS in the Makefile. For example:
CSS=--template-option=css=style.css --template-options=css=site.cssThis will enable two CSS files in the generated HTMLs: style.css and site.css.
HTML templates are supported since Pandoc 1.4. Indeed, the web page you see on this site are completed with TOC sidebar, footer, and disqus comment systems. This is achieved by using a custom template, which can be enabled by changing the OPTIONS setting in the Makefile:
OPTIONS=$(CSS) --template=html.templateThis will make all generated HTMLs from a template file called html.template in the directory where you run make. Template variables can also be set by adding --template-option="var=value" to the OPTIONS setting. If you have installed Pandoc 1.4, a sample template can be found at ~/.cabal/share/pandoc-1.4/templates/html.template. More details on how to use templates are given in the Pandoc User’s Guide.
You may copy any generated HTML file manually to a website to publish, or you can use Make to automate this process. You may have already noticed that in the Makefile we have a PUBS variable. Just put all the files you want to make public in the PUBS variable, and the rest in the DOCS variable after $(PUBS). For example, part of our Makefile may look like this:
PUBOPTIONS=--template-option="author=Some Author"
PUBS=index.html album.html
DOCS=$(PUBS) diary.htmlThe above configuration defines in total three documents, index.html, album.html, and diary.html, where the first two are classified as public. Regardless of public or private, all HTML files are viewable from the local folder.
Also note that in the above example we have PUBOPTIONS setting the Pandoc template variable author to Some Author, which is a means to provide extra settings for public documents when we publish them to websites. Doing just make will only prepare all documents locally without using PUBOPTIONS. Public documents are prepared differently by using:
make publicThis will generate all public HTMLs using the extra PUBOPTIONS (in addition to existing OPTIONS), and put them in a subfolder called public, ready to be uploaded to websites or copied to Blogs. Private documents are not affected during this make.
To further automate the entire publishing process, we can put commands to the public target in the Makefile, such as uploading files from the public subfolder to a FTP, or sync to a WebDAV folder, or things like that. Details are not discussed here.
Here is how to automate the Make process in TextWrangler, an excellent free text editor for OS X (or its non-free big brother BBEdit):
tell application "TextWrangler" to set theFile to file of document 1
set htmlFile to do shell script "echo " & theFile & "|sed -e s/.txt$/.html/"
set filePath to POSIX path of theFile
set htmlPath to POSIX path of htmlFile
set dirName to do shell script "dirname " & filePath
set markdown2html to dirName & "/bin/markdown2html "
do shell script "rm -f " & htmlPath
do shell script "make -C " & dirName
do shell script "test -f " & htmlPath & " || " & markdown2html & filePath
tell application "Finder" to open htmlFileAfter this, we’ll be able to see our script appearing as a menu item in TextWrangler’s Script menu. Clicking it will automatically execute the make command in the folder of the current document, and then view the result HTML file in a web browser, which is usually Safari.
Even better, we can assign it a keyboard shortcut:
CTL+OPT+CMD MNote: The above script assumes the document you are editing is saved with a .txt extension, and is already added to the DOCS variable in the Makefile. Otherwise it’ll fall back to a generic translation directly using markdown2html instead of make. The result will be just a basic HTML, and you’ll loose customizations in the Makefile for things like CSS, and so on.
The combination of pandoc + make + darcs is very flexible, but still too technical to be used by a non tech-savvy user. I plan to evolve it into a single tool, with integrations to popular editors and browsers.
A more immediate goal is to support multiple users and site syncing so that this site can invite more authors.