How to sync documents with Github Pages?

Question

I have a project with several people, and we have a README.md file with a bunch of GitHub Flavored Markdown, which is displayed on our GitHub page. We also created the GitHub Pages branch, which is located under our GitHub Organization subdomain, and used the Automatic Page Generator by simply loading it into our README.md when we created our page. However, I notice that updating our README.md file does not refresh the project page. Instead, we should go to the GitHub settings tab and recreate the project page by reloading the README.md file when we do this.

Also, after reading the relative binding working between the documentation files on the pages of the GitHub project directory. I really like markdown, as it saves a ton of time when you need to write an entire HTML file for our documentation. However, I would like to have one README.md file, which may include relative links to other documentation files located in docs/*.md . I was hoping there was a simple solution so that my other documentation files could also be included in my gh-pages branch and hosted under my GitHub Pages subdomain and be rendered and / or theme.

In other words, my questions are:

• Is there a way to automatically update my README.md file on my Github subdomain?
  • [EDIT]: There is no answer if you use the automatic page generator. You must go to the settings page for the repo and reload it every time there is a change to update it.
    
• Is there a way for my relative links to my documentation to my README.md file to work on my Github pages, maybe somehow synchronize my /docs/*.md with my Github pages and somehow render and / or render them their subjects?
  • [EDIT]: From what I learned after writing this question, it seems that this is only possible on GitHub pages using a static site generator like the Jekyll ruby stone and possibly some uses of the web hosts supported by GitHub that are mentioned in the comments below. I am currently trying to find the best solution.
    
• Even better, is there an even simpler way to do this and possibly have only one copy of my README.md and documentation that is used both on gh pages and on my main branch and makes everything simpler?
  • [EDIT]: It seems almost certainly not. I was thinking about the possibility of something built into GitHub to allow this. It looks like future support for G-Hub Pages may be better provided in the future, or at least I definitely hope it will.

Answer 1

I am going to post a solution that I am setting up to take advantage of the fact that GitHub Pages is already using Jekyll with an automatic page generator.

• git checkout gh-pages
• mkdir _layouts
• mv index.html _layouts
• git checkout master -- README.md
• mv README.md index.md
• Prepare the following text for index.md

 --- layout: index --- 

You also need to open the index.html file and make the following changes:

• Remove the displayed HTML from the markdown in your README.md file. This usually happens between the <section> or <article> tags. Replace this HTML with the text {{ content }} , this will allow us to use this file as jekyll. The file to which we apply the layout will be placed where the content tag is located.

• Find CSS for the project page theme. for me it was a line like the following:

  <link rel='stylesheet' href='stylesheets/stylesheet.css' />

  It needs to be changed to

  <link rel='stylesheet' href='{{ site.path }}/stylesheets/stylesheet.css' />

• Any other assets stored on your site that will be used in this layout must also be prefixed with {{ site.path }} .

Having done this, Jekyll will display the markup file as the contents of the index.html layout in the _layouts directory. To automate this process not only for the README.md file, but also for other documents that may be in your main branch, I took the following steps:

Created a file called post-commit containing the following:

 #!/bin/bash ### ### The following block runs after commit to "master" branch ### if [ `git rev-parse --abbrev-ref HEAD` == "master" ]; then # Layout prefix is prepended to each markdown file synced ################################################################### LAYOUT_PREFIX='---\r\nlayout: index\r\n---\r\n\r\n' # Switch to gh-pages branch to sync it with master ################################################################### git checkout gh-pages # Sync the README.md in master to index.md adding jekyll header ################################################################### git checkout master -- README.md echo -e $LAYOUT_PREFIX > index.md cat README.md >> index.md rm README.md git add index.md git commit -a -m "Sync README.md in master branch to index.md in gh-pages" # Sync the markdown files in the docs/* directory ################################################################### git checkout master -- docs FILES=docs/* for file in $FILES do echo -e $LAYOUT_PREFIX | cat - "$file" > temp && mv temp "$file" done git add docs git commit -a -m "Sync docs from master branch to docs gh-pages directory" # Uncomment the following push if you want to auto push to # the gh-pages branch whenever you commit to master locally. # This is a little extreme. Use with care! ################################################################### # git push origin gh-pages # Finally, switch back to the master branch and exit block git checkout master fi 

EDIT: I updated the above script for the README.md file and markdowns in docs/* to use the same layout file. This is much better than before. This script is in your .git/hooks/ directory. bash should be in your path.

Create a _config.yml file with the following

 markdown: redcarpet path: http://username.imtqy.com/reponame 

The aforementioned script also synchronizes the markup files found in the docs/* directory of the master branch so that they can be viewed on the GitHub Pages website. Relative binding to these documents works if you include the following jQuery function to separate the .md extension from the anchors on the gh-pages branch. You can add the following script in index.html to the _layouts directory:

 $(document).on('ready', function () { $('a').each(function (i, e) { var href = e.href; if (href.search('.md') > 0) $(this).attr('href', href.split('.md')[0]); }); }); 

EDIT: I changed the code above in my repository, it was a quick and dirty way to do this, but it will not work correctly in all cases if you know what I mean. For example, the markdown file company.mdata.md will not be processed correctly. To fix this, I updated it to the next script, which checks href more carefully and removes the extension if it is found. I also made the script more general, allowing you to use it to remove other extensions by changing the ext variable. Here is the code:

 $(function () { $('a').each(function () { var ext = '.md'; var href = $(this).attr('href'); var position = href.length - ext.length; if (href.substring(position) === ext) $(this).attr('href', href.substring(0, position)); }); }); 

I am installing a repo example in CoryG89 / docsync , which has a project page here if you want to see how it all works together.

Answer 2

My solution to the README synchronization issue with the Github page is slightly different from the previous one. Instead of using a separate Markdown JavaScript engine, you can use the Github API to return a Markdown file that displays as HTML.

• Extract README.md from https://api.github.com/repos/<owner>/<repo>/contents/README.md .
• Decode Base64 response: window.atob( JSON.parse( blob ).content );

• Put decoded README in https://api.github.com/markdown in JSON body

    { "text": "<README>", "mode": "markdown", "context": "<owner>/<repo>" } 

• Insert the rendered HTML into the DOM element, as done by Brad Rhodes .

Two caveats to this approach:

• Running two consecutive requests slows down the page loading.
• You may see a speed limit when accessing the Github API.

For a page with low traffic, where the load time is not critical (~ 1-2 sec.), This method works quite well.

Answer 3

I have a couple of ideas for sharing a single readme file between your documentation site and the main github repository:

• You can use only one branch of gh pages, which contains both your code and the jekyll documentation site. Your repository may be a little cluttered, and you will need to put the YAML header at the top of the readme. It almost supports relative snapping. The problem is that if you want jekyll to display your markdown, it will provide it with the .html extension. Maybe there is a way to tweak this though. Here is an example that I put together to see if it works.

• You can use the AJAX calls on your documentation site to read the readme from your main branch and then render it using the Javascript Markdown renderer . It will take a little longer to load and will not support relative links unless you write some clever Javascript It is also more work to implement than idea 1.

Answer 4

Another way to consider this is to set up a pre-commit hook that builds the appropriate pages. I do this in one of my repositories . You probably have to turn off the automatic page generator and just click on the gh-pages branch yourself, though, and also do something fantastic to turn your documents into HTML or the Jekyll website as Nathan suggests .

In this repository, I click to keep gh-pages identical to master . There are many other ways to do this. This may not be ideal for your situation, though (perhaps you do not want them to be the same).

In any case, the reason I offered generosity on this issue was because I was hoping that someone would have a better workflow. This method is kind of tortuous and inflexible, and it requires everyone to keep their hooks in sync.

Answer 5

Another way I've worked with successfully is to use Ajax to retrieve documents using the Github API and the Javascript markup engine for rendering HTML (as Nathan also suggests).

• Use the Github API and JSONP to fetch a document from Github
• Decode base64 content in Github API response
• Mark markdown using javascript markup mechanism.
• Display displayed html

Nathan expressed some concern about performance, but in my experience it loads, it would seem, instantly, so I don’t think this is actually a problem.

The advantage is that it is easily customizable and it will always update your documents, even if you simply edit the markdown directly in the browser on github.

I installed the example on the Github pages at http://bradrhodes.imtqy.com/GithubDocSync/ to show that it works.

Answer 6

You can use DocumentUp to render your README.md.

Answer 7

Another possibility for the method described by Nathan and Brand Rhodes is to use a great tool: FlatDoc , created by Rico Sta. Cruz.

FlatDoc will download ajax documentation (README.md or any other markdown file), analyze it and display it with all the goodies and even the sidebar menu for navigation!

He created in his api a helper method for downloading files from the GitHub replica master server (but can also download to another location from the Internet).

Instruction manual

Start by copying the following html template to your index.html in the gh-pages branch. Continue:

• Replacing "USER" with your GitHub Username
• Replacing "REPO" with your GitHub repo name
• Replacing "your project" with your project name

in file. Try it locally in your browser. Then commit and push changes. Now your github page will be constantly updated with your README.md file in your main branch.

If the default theme is not suitable for you, you can redo it using your own css.

Answer 8

I also want to edit documents in master and publish in gh pages - I like to keep documents up to date with the source code, and this seems like the best way. This is an unfinished business for me, but I took the Cory script as a starting point and expanded it a bit so that it works out of the box while it is a branch of gh pages with _layouts (i.e. jekyll site). It converts fencing backward style fencing (for code blocks) that work great in github view mode but not on gh pages. I am using index.md with include for the README.md project, so I can add a title and some other decorations. This version also processes documentation in any subdirectories called "docs", which I find useful in a project with several modules (no git submodules, only subdirectories):

.git/hooks/post-commit

 #!/bin/bash ### ### The following block runs after commit to "master" branch ### if [ `git rev-parse --abbrev-ref HEAD` == "master" ]; then # function to convert a plain .md file to one that renders nicely in gh-pages function convert { # sed - convert links with *.md to *.html (assumed relative links in local pages) # awk - convert backtick fencing to highlights (script from bottom of file) sed -e 's/(\(.*\)\.md)/(\1.html)/g' "$1" | awk -f <(sed -e '0,/^#!.*awk/d' $0) > _temp && mv _temp "$1" } if ! git show-ref --verify --quiet refs/heads/gh-pages; then echo "No gh-pages, so not syncing" exit 0 fi # Switch to gh-pages branch to sync it with master ################################################################### git checkout gh-pages mkdir -p _includes # Sync the README.md in master to index.md adding jekyll header ################################################################### git checkout master -- README.md if [ -f README.md ]; then cp README.md _includes/ convert _includes/README.md git add README.md git add _includes/README.md fi # Generate index if there isn't one already ################################################################### if [ ! -f index.md ]; then echo -e '---\ntitle: Docs\nlayout: default\n---\n\n{% include README.md %}' > index.md git add index.md fi # Generate a header if there isn't one already ################################################################### if [ ! -f _includes/header.txt ]; then echo -e '---\ntitle: Docs\nlayout: default\nhome: \n---\n\n' > _includes/header.txt git add _includes/header.txt fi # Sync the markdown files in all docs/* directories ################################################################### for file in `git ls-tree -r --name-only master | grep 'docs/.*\.md'` do git checkout master -- "$file" dir=`echo ${file%/*} | sed -e "s,[^/]*,..,g"` cat _includes/header.txt | sed -e "s,^home: .*$,home: ${dir}/," > _temp cat "$file" >> _temp && mv _temp "$file" convert "$file" git add "$file" done git commit -a -m "Sync docs from master branch to docs gh-pages directory" # Uncomment the following push if you want to auto push to # the gh-pages branch whenever you commit to master locally. # This is a little extreme. Use with care! ################################################################### # git push origin gh-pages # Finally, switch back to the master branch and exit block git checkout master fi exit $? #!/usr/bin/awk { # Replace backtick fencing (renders well when browsing github) with jekyll directives if (/```/) { IN = IN?0:1 # Are we already in a fenced section? Toggle. if (IN) { # If we are starting a fenced section if (/```\s*$/) { $0 = $0"text" # empty language is OK for backticks but not for jekyll } gsub(/```/, "{% highlight ") print $0" %}" } else { # ending a fenced section print "{% endhighlight %}" } } else { # not a fencing line if (IN) { # but in a fenced section, so add indent to make sure code is rendered with <pre> print " "$0 } else { print } } } 

Another variation of the original is that it sets the page.home variable on all pages. This can be used to determine the relative path of the root crop, so it can be used to search for static resources such as css. In _layouts/.default.html I have:

 <link rel="stylesheet" href="{{ page.home }}css/main.css"> 

Thus, I can edit css, create the jekyll site locally and see the result in the browser, without waiting for github to build it on the server.

Answer 9

I recently made the gh-pages-generator package to solve this problem - it creates a multi-page site using several MD files and a configuration file.

It correctly updates all links between pages. It is relatively easy to make it part of CI to commit changes to gh-pages branches.

I use here and here .

Answer 10

It’s not difficult , two copies and pastes into the terminal, and you are all set up.

Jekyll lets you import your markup file, and then it takes care of converting them to HTML. The trick is to import README.md into your index.md file using {% include_relative README.md %} . Here's how we can do it:

It’s worth checking out how to set up the Jekyll superbog site on github (these are just two files! >)

Customization

You can copy these two files and go to the page with your current readme, just by running this setting once (copy the entire code block and execute pase to the terminal):

 # Copy our two files to the gh-pages branch git checkout -b gh-pages && wget https://raw.githubusercontent.com/lazamar/barebones-jekyll-project-readme/master/_config.yml && wget https://raw.githubusercontent.com/lazamar/barebones-jekyll-project-readme/master/index.md && # # Commit and publish our page on github git add -A && git commit -m "Create project github page" && git push --set-upstream origin gh-pages | # git checkout master # go back to master branch 

Automating

Then we just need to automate the task of copying all changes from master to the gh-pages branch before each click. We can do this by running this script (you can copy and paste it into the terminal)

 $(cat > .git/hooks/pre-push << EOF #!/bin/sh we_are_in_gh_pages="\$(git branch | grep -G "* gh-pages")" if [ ! "\$we_are_in_gh_pages" ]; then git checkout gh-pages && git rebase master && git push -f && git checkout master # go back to master branch fi EOF ) && chmod 775 .git/hooks/pre-push 

It will create a trigger that will copy all changes from the master branch to gh-pages every time git push run.

What is it. Done.