When I started learning perl, ages ago, it really opened up a lot of new intellectual opportunities for me. With perl, if I had a text stream, I could parse and extract from it -- and everything was text then! I'm hoping Python will be an "enabling" kind of thing like that too, but up to date for the now. Here goes....
Sunday, December 19, 2010
Learning Python (Fourth Edition)
When I started learning perl, ages ago, it really opened up a lot of new intellectual opportunities for me. With perl, if I had a text stream, I could parse and extract from it -- and everything was text then! I'm hoping Python will be an "enabling" kind of thing like that too, but up to date for the now. Here goes....
Tuesday, December 7, 2010
Linux, Swap and Render Farms
The debate occurred again last night, with a Cinematics group, about how best to configure their linux render nodes. We're running a bare metal hypervisor, with two guests running RHEL 5, on a pair of xeon 5520 processors, with 12 Gb RAM and a slow-mo SATA disk. How should we allocate RAM to the guests, and do we want swap?
Sven immediately offered this article on how mysql behaves on a NUMA CPU architecture, which suggests to me that I might as well turn swap off. The counterpoint (and traditional view) is that a linux machine must have swap, as a part of its normal modus operandi. So far, the render nodes show no difference with/without swap (so I turned it off) and two virtual guests outperform one OS running on bare metal. I suspect the virtualization layer is sticking one guest on each of the processors, so their RAM is staying local to the CPU they're on.
I don't know the answer, but it's gonna be interesting. If you're in the same boat, you'll appreciate this handy secret-decoder-ring for vmstat. And here's a primer on "swappiness." Also, here's the middle road solution, enable swap, but just a pittance of it.
Finally, if you're running a render/compute farm, you'll find these bits handy ... there's a lovely o'reilly article on how the linux out-of-memory killer works. You'll also be interested in this article on how linux memory overcommit works. And finally, if you're trying with and without swap, here's a quick bit on how to use a normal file as swap, so you donut have to repartition anything to test the on/off configs.
Hopefully this helps someone!
Sven immediately offered this article on how mysql behaves on a NUMA CPU architecture, which suggests to me that I might as well turn swap off. The counterpoint (and traditional view) is that a linux machine must have swap, as a part of its normal modus operandi. So far, the render nodes show no difference with/without swap (so I turned it off) and two virtual guests outperform one OS running on bare metal. I suspect the virtualization layer is sticking one guest on each of the processors, so their RAM is staying local to the CPU they're on.
I don't know the answer, but it's gonna be interesting. If you're in the same boat, you'll appreciate this handy secret-decoder-ring for vmstat. And here's a primer on "swappiness." Also, here's the middle road solution, enable swap, but just a pittance of it.
Finally, if you're running a render/compute farm, you'll find these bits handy ... there's a lovely o'reilly article on how the linux out-of-memory killer works. You'll also be interested in this article on how linux memory overcommit works. And finally, if you're trying with and without swap, here's a quick bit on how to use a normal file as swap, so you donut have to repartition anything to test the on/off configs.
Hopefully this helps someone!
Labels:
sysadmin
Location:
Sorrento Valley, San Diego, CA, USA
Sunday, December 5, 2010
What Domains did ICE Shut Down?
I was curious; which actual sites did the Attorney General shut down? Well each seized domain has had its name servers' IPs replaced with 74.81.170.109 and 74.81.170.108. So the closest I could get, was to use a tool to find the A records that aim at a particular IP and see all the names those IPs are going by. (Looking up the PTR doesn't work, so you gotta be a little sneaky.)
If you do this on NS1, you get a list of ns1.somedomain.tld entries in the resulting web page, and if you copy and paste the list into a text file you can clean it up like this
` cat textfile | awk {'print $1'} | grep -i ns1 `
Which currently (Sunday, December 5, 2010, 23:30 GMT) results in only thirty seized domains' name servers and ns1.seizedservers.com itself:
ns1.2009jerseys.com
ns1.51607.com
ns1.amoyhy.com
ns1.bishoe.com
ns1.borntrade.com
ns1.boxedtvseries.com
ns1.boxset4less.com
ns1.burberryoutlet-us.com
ns1.burberryoutletshop.com
ns1.cartoon77.com
ns1.coachoutletfactory.com
ns1.dajaz1.com
ns1.dvdorderonline.com
ns1.dvdprostore.com
ns1.handbag9.com
ns1.handbagcom.com
ns1.jersey-china.com
ns1.lifetimereplicas.com
ns1.louis-vuitton-outlet-store.com
ns1.mydreamwatches.com
ns1.oebags.com
ns1.realtimberland.com
ns1.rmx4u.com
ns1.seizedservers.com
ns1.skyergolf.com
ns1.sunogolf.com
ns1.throwbackguy.com
ns1.tieonsale.com
ns1.topabuy.com
ns1.torrent-finder.com
ns1.usaoutlets.net
I am definitely doing something wrong -- that's significantly less than the 82 being reported by most bloggers. Naturally, the government has no comments on what domains they've seized. (I'm not sure I like the sekrit-p0lice implications of that.) Also interesting to note, they're running both google analytics and piwik on their take-down notice page.
If you do this on NS1, you get a list of ns1.somedomain.tld entries in the resulting web page, and if you copy and paste the list into a text file you can clean it up like this
` cat textfile | awk {'print $1'} | grep -i ns1 `
Which currently (Sunday, December 5, 2010, 23:30 GMT) results in only thirty seized domains' name servers and ns1.seizedservers.com itself:
ns1.2009jerseys.com
ns1.51607.com
ns1.amoyhy.com
ns1.bishoe.com
ns1.borntrade.com
ns1.boxedtvseries.com
ns1.boxset4less.com
ns1.burberryoutlet-us.com
ns1.burberryoutletshop.com
ns1.cartoon77.com
ns1.coachoutletfactory.com
ns1.dajaz1.com
ns1.dvdorderonline.com
ns1.dvdprostore.com
ns1.handbag9.com
ns1.handbagcom.com
ns1.jersey-china.com
ns1.lifetimereplicas.com
ns1.louis-vuitton-outlet-store.com
ns1.mydreamwatches.com
ns1.oebags.com
ns1.realtimberland.com
ns1.rmx4u.com
ns1.seizedservers.com
ns1.skyergolf.com
ns1.sunogolf.com
ns1.throwbackguy.com
ns1.tieonsale.com
ns1.topabuy.com
ns1.torrent-finder.com
ns1.usaoutlets.net
I am definitely doing something wrong -- that's significantly less than the 82 being reported by most bloggers. Naturally, the government has no comments on what domains they've seized. (I'm not sure I like the sekrit-p0lice implications of that.) Also interesting to note, they're running both google analytics and piwik on their take-down notice page.
Friday, December 3, 2010
Subscribe to:
Posts (Atom)