for keeping track of my digital experiments

I had first heard about the Warehouse subversion browser a couple of months back. At the time it was still closed source and cost $30 per server license. Recently though, it went open source and is now freely available. For those of you with about 1-2 hours of time and know how to setup and run subversion on a Linux/Unix box (with WebDAV) it really isn’t that hard to get up and running.

I installed this on one of my home Debian Etch (4) servers and it took me about 30 minutes once I understood the entire process.

1. Install some necessary dependencies for building (I prolly left out a few) – make, rake, unzip, zip, tar, gzip, bzip2, libc6, gcc, cpp, g++
2. Install Ruby
   • download the latest tarball (source code in tar.gz) here – http://www.ruby-lang.org/en/downloads/ – wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz
     
   • run – tar xzvf ruby*.tar.gz – to open up the archive
   • cd into the directory – cd ruby*
   • configure, make and then install –> 1) ./configure 2) make 3) make install (as root)
3. Install Swig
   • download one of the latest tarballs – wget http://internap.dl.sourceforge.net/sourceforge/swig/swig-1.3.31.tar.gz
   • unarchive the archive – tar xzvf swig-1.3.31.tar.gz
   • get into the directory – cd swig-1.3.31
   • configure, make, then install –> 1) ./configure 2) make 3) make install (as root)
4. Install Subversion and the Swig-Ruby bindings
   • download one of the latest subversion archives – wget http://subversion.tigris.org/downloads/subversion-1.5.5.tar.gz
   • unarchive that sucker – tar xzvf subversion*.tar.gz
   • get into the directory that was just created – cd subversion*
   • configure for installation (and if you are missing anything be sure to install it!) – ./configure –enable-shared –enable-static –enable-debug –with-ssl –with-swig
   • make and install –> 1) make 2) make install
   • now install the swig-ruby bindings –> 1) make swig-rb 2) make install-swig-rb
5. Install various dependencies for later: apache2 with WebDAV support, ruby gems, MySQL server/client, PHPMyAdmin might be handy as well
6. Download the Warehouse source code – http://github.com/entp/warehouse/tree/master
7. Untar into where you want Warehouse to start from: tar xvzf, mv, and whatnot.
8. Setup a 3 MySQL databases (warehouse_test, warehouse_production, warehouse_dev) that can be accessed by 1 user for testing, production and development.
9. Now run the warehouse bootstrapping program and setup procedures. (No registration key needed now that it is open source, so leave that field blank.) – http://www.warehouseapp.com/installing/installing-and-registering-warehouse
10. In the Warehouse settings, be sure to setup warehouse so that it updates your WebDAV permissions and passwords.
    • Shell command to run when permissions are updated. This will auto-generate the necessary permission file for WebDAV: rake warehouse:build_config CONFIG=[warehouse directory]/config/access.conf
    • Shell command to run when someone’s password is updated. This will auto-generate the necessary password file for WebDAV: rake warehouse:build_htpasswd CONFIG=[warehouse directory]/config/htpasswd.conf

This year I am enrolled in the PSEO program at Century college. To keep up with what is going on at my high school, I subscribed to the Woodbury High School events mailing list. Oddly enough, the host it was coming from was none other than a host in the domain of trend micro. So I did a quick DNS query and found out that the South Washington County schools (District 833) are using an off site hosted email solution from Trend Micro called, InterScan™ Messaging Hosted Service.

blackbox:~# host --anything sowashco.k12.mn.us
sowashco.k12.mn.us A 63.225.128.3
sowashco.k12.mn.us MX 10 in.sjc.mx.trendmicro.com
!!! sowashco.k12.mn.us MX host in.sjc.mx.trendmicro.com is not canonical
sowashco.k12.mn.us NS authns1.mpls.qwest.net
sowashco.k12.mn.us NS authns2.dnvr.qwest.net
sowashco.k12.mn.us NS authns3.sttl.qwest.net
sowashco.k12.mn.us SOA authns1.mpls.qwest.net dns-admin.qwestip.net (
2007092800 ;serial (version)
10800 ;refresh period (3 hours)
3600 ;retry interval (1 hour)
604800 ;expire time (1 week)
86400 ;default ttl (1 day)
)

I had been meaning to check into District 833′s (url filtration) methods for quite some time, so I would be able to figure out similar ways of imaging and control of the six (operational) machines on my Local Area network.

The statistics on their network, according to a testimonial posted on Trend Micro’s website:

• fiber optic switches that connect all 24 school district sites together
• 3,000+ workstations
• 30,000 spyware and virus accomplice hits every day (during the school year)
• 650,000 messages a day about 640,000 of those messages are spam

[Dietsche] continued, “When any end user logs on to a workstation, InterScan Web Security Suite checks the Active Directory to verify that the user is authorized to be on the network. Verification is instantaneous and scanning begins without the end user even knowing it.” Dietsche also can track events by user accounts. “This is a huge advantage for the IT staff,” he said. “In the past, we found it difficult to isolate a security issue. But now, LDAP authentication enables us to get log reports that identify who the affected users are and which of them are constantly running into viruses.”

I thought that was way cool how all of these solutions can come together in one cohesive structure.

I was originally looking around for a preexisting methodology of peer to peer networking that would suit my upcoming game engine. It needed to be able to function without a central server, have low peer lookup times, and the network had to be resilient to outages. As usual I was sidetracked by interesting forms of peer to peer architectures. I found two interesting articles which I shall highlight.

Overnet
I discovered the Overnet two years too late. It was an interesting lead, a preexisting implementation of DHT methodology, combined with the now defunct eDonkey network. The Overnet was a true serverless network. Nodes that wanted to connect would query a preexisting peer list or get new initial peers via the eDonkey net. The Overnet-ish system is still in use today, but for other things than file sharing.

Storm botnet
One of the most prevalent uses of the Overnet still in use today, is the Storm botnet. It is a remotely-controlled network of “zombie” computers (or “botnet”) that has been linked by the Storm Worm, a Trojan horse spread through e-mail spam (Wikipedia). The size of it is staggering, with conservative estimates at 160,000 to a more agreed upon 1 million. The crazy part is the amount of bandwidth that 1 million geo-diverse DSL lines could throw at a host. 1 million computers * a mid-range DSL line (256kbps) = 244.14 Gbps = a whole lot of bandwidth. I am not really scared, because I don’t pick fights with multinational spam corporations. What I find amazing is how peer to peer architectures can scale so much.