There are lots of available solutions to update files remotely, [S]FTP, git/svn, etc.

I have personally used FTP, tried out git and even attempted to used macFuse to mount a local drive of an SFTP folder.

FTP is ok for one file at a time but becomes unusable for many files, git is acceptable for multiple file changes, keeps history but is a bit too slow for one file changes and macFuse is impossibly slow to mount.

I set out to find the silver bullet and went to a place long forgotten.
Enter the unix world, the world where programs do one thing and do it well. rsync. It’s purpose is to keep directories in sync, the largest mirror sites use this to keep up to date with the changes.
In my setup I am using rsync over an ssh connection, which makes this a tad be more complicated, still extremely useful while safe.

Keeping local directories in sync with rsync is pretty simple.

> rsync origin/ destination/

That will keep the contents of the directory ‘destination’ in sync with ‘origin’.
I added the option -r to make sure rsync recurses into directories.

> rsync -r origin/ destination/

rsync does not delete files from destination that are missing in origin folder, for this i added –delete and –force to remove directories from destination even if they are not empty.

> rsync -r –delete –force origin/ destination/

But because I sometimes work on live code the updates should not mess with the running code. For this I added the option –delay-updates and changed –delete to –delete-after. –delay-updates will tell rsync to upload all changes and apply them after all uploads are sent, making the time it takes to apply changes much shorter. –delete-after will tell rsync to delete the files after uploading. These both options make changes look more “atomic”.

> rsync -r –delete-after –delay-updates –force origin/ destination/

Because I get bored really fast and having some info on how things are going I also added –progress, this will show how much has been uploaded, transfer speed and ETA.

> rsync -r –delete-after –delay-updates –force –progress origin destination

Know because sync happens over the internet I decided to use ssh for the transfer. To tell rsync to transfer over ssh I used the -e option which allows the user to define a shell for rsync.

> rsync -r –delete-after –delay-updates –force –progress -e ’ssh’ origin/ example.com:~/destination/

Note to separate the domain name from the path with ‘:’

Most of the work is done, but has you may notice, if you have large files, the upload speed is not very fast. To help speed up the syncing I use -z

> rsync -r –delete-after –delay-updates –force –progress -e ’ssh’ -z origin/ example.com:~/destination/

or for a shorter version

> rsync -rze ’ssh’ –force –delete-after –delay-updates –progress origin/ example.com:~/destination/

At this point rsync is pretty decent, except it request the password every time you want to sync.

This is easily fixed with public/private key files.

For further help take a look at this fine tutorial on setting up ssh keys.

I wanted to have a git based repository of wordpress but the most recent one I found in github was skynet’s one.

This repo had been last updated to r11948 in September 2009 so i had to either

• fetch the whole history of wordpress from the svn repo since sometime around 2006,
• clone svn and lose the history or
• clone this git repo add the remote svn  from wordpress.org and then update since r11948

The first was out of question has it is both unecessary and useless, it would also take a shit load of time to complete. The second was something I did not want to resort to has i wanted to keep connected to skynet and it’s other forks inside github. I had to try to clone from github and then somehow connect to the wordpress svn repo.

I have been fiddling around with this for the last few days and has I expected, in the end, it was quite simple to do.

First I forked skynet repository to create one of my own and keep the relation inside of github. This is somewhat unecessary has i could have just cloned skynet repo directly but i wanted to keep the github network connection.

With the github repo forked I cloned the svn repo from wordpress.org from the point in which it was left by skynet.

For this I looked into the information of the last update and took the revision number 11948.

> git svn clone http://core.svn.wordpress.org/trunk/ wordpress -r 11949

This will tell git to clone the svn repository starting after the last revision.

Now the repo needs to be brought up to date.

> cd wordpress

> git svn rebase

This will get all revisions past the current HEAD and rebase all changes

Then I added my github repository to the local configuration.

> git remote add github git@github.com:medecau/wordpress.git

And pushed to github

> git push github

I also cleaned up, has svn tends to be quite messy.

> git gc

And now I have fresh WordPress repository in  github

Thanks in part to fnokd.

Just got my invite to Google Voice, can’t use it because it’s not available in my country,

Next time let me know when I ask for the invite, thanks.

This got me curious again and knowing, from previous readings, that bittorrent trackers use HTTP the idea for trackhub started to grow on my head. I started to wonder if BT clients followed HTTP redirects.After searching the web for bittorrent+redirect, I found that some BT clients used standard libraries and thus had this already built in.

The plan was in motion, I created the app instance on GAE and started coding. A few tests later and sure enough some guys were able to download. I started to ask some questions on irc.freenode.net/#bittorrent and fixed some bugs and functionality, released it to github and fired an email to torrentfreak.com editor Ernesto. After the article on torrentfreak traffic immediately spiked and a few bug fixes had to be done.

But soon enough it was obvious that trackhub would not survive.

Why?

There’s a few reasons for this:

• trackhub sucks, help it suck lesss
• keeps logs and is US based
• GAE resources are limited
• Popular bt clients misbehave
• .torrent creators put trackhub next to other trackers

But the idea is not completely stupid. In my opinion it helps shed light on some problems with BT.

What’s needed?

• More public trackers that can be trusted
• Bt clients that speak proper http
• pirates that walk the walk, develop and/or donate to these projects
• .torrent creators use proper tools to create the .torrent files and use tiers
• UDP trackers, DHT, PEX, etc.