toolshed

now use leveldb databases to track processed sources

make explore not follow symlinks and check if file is symlink before hashing the file