TechOpsGuys.com Diggin' technology every day

December 12, 2010

OpenBSD installer: party like it’s 2000

Filed under: linux,Random Thought,Security — Tags: , , — Nate @ 12:07 am

[Random Thought] The original title was going to be “OpenBSD: only trivial changes in the installer in one heck of a long time” a take off of their blurb on their site about remote exploits in the default install.

I like OpenBSD, well I like it as a firewall — I love pf. I’ve used ipchains, iptables, ipfwadm, ipf (which I think pf was originally based off of and was spawned due to a licensing dispute with the ipf author(s)), ipfw, Cisco PIX and probably one or two more firewall interfaces, and pf is far and away the best that I’ve come across.  I absolutely detest Linux’s firewall interfaces by contrast, going all the way back almost 15 years now.

I do hate the OpenBSD user land tools though, probably as much as the *BSD folks hate the Linux user land tools. I mean how hard is it to include an init script of sorts to start and stop a service? But I do love pf, so in situations where I need a firewall I tend to opt for OpenBSD wherever possible (when not possible I don’t resort to Linux, I’d rather resort to a commercial solution perhaps a Juniper Netscreen or something).

But this isn’t about pf, or user land. This is about the OpenBSD installer. I swear it’s had only the most trivial changes and improvements done to it in at least the past 10 years, when I first decided to try it out. To me it is sad, the worst part about it is of course the disk partitioning interface. It’s just horrible.

I picked up my 2nd Soekris net5501 system and installed OpenBSD 4.8 on it this afternoon, and was kind of sadened, yet not surprised how it still hasn’t changed. I have my other Soekris running OpenBSD 4.4 and has been running for a couple years now. First used pf I believe back in about 2004 or so, so have been running it quite a while, nothing too complicated, it’s really simple to understand and manage. My first experience with OpenBSD was I believe back in 2000, I’m not sure but I want to say it was something like v2.8. I didn’t get very far with it, for some reason it would kernel panic on our hardware after about a day or so of very light activity, so went back to Linux.

I know pf has been ported to FreeBSD, and there is soon to be a fully supported Debian kFreeBSD distribution with the next major release of Debian whenever that is, so perhaps that will be worth while switching to for my pf needs, I don’t know. Debian is another system which has been criticized over the years for having a rough installer, though I got to say in the past 4-5 years it really has gotten to be a good installer in my opinion. As a Debian user for more than 12 years now it hasn’t given me a reason to switch away from it, but I still do prefer Red Hat based distros for “work” stuff.

First impressions are important, and the installer is that first impression. While I am not holding out hope they will improve their installer, it would be nice.

October 8, 2010

Manually inflating the memory balloon

Filed under: Virtualization — Tags: , — Nate @ 12:10 am

As I’m sure you all know, one of the key technologies that VMware has offered for a long time is memory ballooning to free memory from idle guest OSs in order to return that memory to the pool.

My own real world experience managing hundreds of VMs in VMware has really made me want to do one thing more than anything else:

Manually inflate that damn memory balloon

I don’t want to have to wait until there is real memory pressure on the system to reclaim that memory. I don’t use windows so can’t speak for it there, but Linux is very memory greedy. It will use all the memory it can for disk cache and the like.

What I’d love to see is a daemon (maybe vmware-tools even) run on the system monitoring system load, as well as how much memory is actually used, which many Linux newbies do not know how to calculate, using the amount of memory reported being available by the “free” command or the “top” command is wrong. True memory usage on Linux is best calculated:

  • [Total Memory] – [Free Memory] – [Buffers] – [Cache] = Used memory

I really wish there was an easy way to display that particular stat, because the numbers returned by the stock tools are so misleading. I can’t tell you how many times I’ve had to explain to newbies that just because ‘free’ is saying there is 10MB available that there is PLENTY of ram on the box because there is 10 gigs of memory in cache. They say, “oh no we’re out of memory we will swap soon!”. Wrong answer.

So back to my request. I want a daemon that runs on the system, watches system load, and watches true memory usage, and dynamically inflates that baloon to return that memory to the free pool, before the host runs low on memory. So often VMs that run idle really aren’t doing anything, and when your running on high grade enterprise stoage, well you know there is a lot of fancy caching and wide striping going on there, the storage is really fast! Well it should be. Since the memory is not being used(sitting in cache that is not being used) – inflate that balloon and return it.

There really should be no performance hit. 99% of the time the cache is a read cache, not a write cache, so when you free up the cache the data is just dropped, it doesn’t have to be flushed to disk (you can use the ‘sync’ command in a lot of cases to force a cache flush to see what I mean, typically the command returns instantaneously)

What I’d like even more than that though is to be able to better control how the Linux kernel allocates cache, and how frequently it frees it. I haven’t checked in a little while but last I checked there wasn’t much to control here.

I suppose that may be the next step in the evolution of virtualization – more intelligent operating systems that can be better aware they are operating in a shared environment, and return resources to the pool so others can play with them.

One approach might be to offload all of storage I/O caching to the hypervisor. I suppose this could be similar to using raw devices(bypasses several file system functions). Aggregate that caching at the hypervisor level, more efficient.

 

September 23, 2010

Using open source: how do you give back?

Filed under: General,linux,Random Thought — Tags: — Nate @ 10:11 pm

After reading an article on The Register (yeah you probably realize by now I spend more time on that site online than pretty much any other site), it got me thinking about a topic that bugs me.

The article is from last week but is written by the CEO of the organization behind Ubuntu. It basically talks about how using open source software is a good way to save costs in a down(or up) economy. And tries to give a bunch of examples on companies basing their stuff on open source.

That’s great, I like open source myself, fired up my first Slackware Linux box in 1996 I think it was(Slackware 3.0). I remember picking Slackware over Red Hat at the time specifically because Slackware was known to be more difficult to use and it would force me to learn Linux the hard way, and believe me I learned a lot. To this day people ask me what they should study or do to learn Linux and I don’t have a good answer, I don’t have a quick and easy way to learn Linux the way I learned it. It takes time, months, years of just playing around with it. With so many “easy” distributions these days I’m not sure how practical my approach is now but I’m getting off topic here.

So back to what bugs me. What bugs me is people out there, or more specifically organizations out there that do nothing but leach off of the open source community. Companies that may make millions(or billions!) in revenue in large part because they are leveraging free stuff. But it’s not the usage of the free stuff that I have a problem with, more power to them. I get annoyed when those same organizations feel absolutely no moral obligation to contribute back to those that have given them so much.

You don’t have to do much. Over the years the most that I have contributed back have been participating in mailing lists, whether it is the Debian users list(been many years since I was active there), or the Red Hat mailing list(few years), or the CentOS mailing list(several months). I try to help where I can. I have a good deal of Linux experience, which often means the questions I have nobody else on the list has answers to. But I do(well did) answer a ton of questions. I’m happy to help. I’m sure at some point I will re-join one of those lists(or maybe another one) and help out again, but been really busy these past few months. I remember even buying a bunch of Loki games to try to do my part in helping them(despite it not being open source, they were supporting Linux indirectly). Several of which I never ended up playing(not much of a gamer). VMware of course was also a really early Linux supporter(still have my VMware 1.0.2 linux CD I believe that was the first version they released on CD previous versions were download only), though I have gotten tired of waiting for vCenter for Linux.

The easiest way for a corporation to contribute back is to say use and pay for Red Hat Enterprise, or SuSE or whatever. Pay the companies that hire the developers to to make the open source software go. I’m partial to Red Hat myself at least in a business environment, though I use Debian-based in my personal life.

There are a lot of big companies that do contribute code back, and that is great too, if you have the expertise in house. Opscode is one such company I have been working with recently on their Chef product. They leverage all sorts of open source stuff in their product(which in itself is open source). I asked them what their policy is for getting things fixed in the open source code they depend on, do they just file bugs and wait or do they contribute code, and they said they contribute a bunch of code, constantly. That’s great, I have enormous respect for organizations that are like that.

Then there are the companies that leach off open source and not only don’t officially contribute in any way whatsoever but they actively prevent their own employees from doing so. That’s really frustrating & stupid.

Imagine where Linux, and everything else would be if more companies contributed back. It’s not hard, go get a subscription to Red Hat, or Ubuntu or whatever for your servers (or desktops!). You don’t have to contribute code, and if you can’t contribute back in the form of supporting the community on mailing lists, or helping out with documentation, or the wikis or whatever. Write a check, and you actually get something in return, it’s not like it’s a donation. But donations are certainly accepted by the vast numbers of open source non profits

HP has been a pretty big backer of open source for a long time, they’ve donated a lot of hardware to support kernel.org and have been long time Debian supporters.

Another way to give back is to leverage your infrastructure, if you have a lot of bandwidth or excess server capacity or disk space or whatever, setup a mirror, sponsor a project. Looking at the Debian page as an example it seems AboveNet is one such company.

I don’t use open source everywhere, I’m not one of those folks who has to make sure everything is GPL or whatever.

So all I ask, is the next time you build or deploy some project that is made possible by who knows how many layers of open source products, ask yourself how you can contribute back to support the greater good. If you have already then I thank you 🙂

Speaking of Debian, did you know that Debian powers 3PAR storage systems? Well it did at one point I haven’t checked recently, I do recall telnetting to my arrays on port 22 and seeing a Debian SSH banner. The underlying Linux OS was never exposed to the user. And it seems 3PAR reports bugs, which is another important way to contribute back. And, as of 3PAR’s 2.3.1 release(I believe) they finally officially started supporting Debian as a platform to connect to their storage systems. By contrast they do not support CentOS.

Extreme Networks’s ExtremeWare XOS is also based on Linux, though I think it’s a special embedded version. I remember in the early days they didn’t want to admit it was Linux they said “Unix based”. I just dug this up from a backup from back in 2005, once I saw this on my core switch booting up I was pretty sure it was Linux!

Extreme Networks Inc. BD 10808 MSM-R3 Boot Monitor
Version 1.0.1.5 Branch mariner_101b5 by release-manager on Mon 06/14/04
Copyright 2003, Extreme Networks, Inc.
Watchdog disabled.
Press and hold the <spacebar> to enter the bootrom.

Boot path is /dev/fat/wd0a/vmlinux
(elf)
0x85000000/18368 + 0x85006000/6377472 + 0x8561b000/12752(z) + 91 syms/
Running image boot…

Starting Extremeware XOS 11.1.2b3
Copyright (C) 1996-2004 Extreme Networks.  All rights reserved.
Protected by U.S. Patents 6,678,248; 6,104,700; 6,766,482; 6,618,388; 6,034,957

Then there’s my Tivo that runs Linux, my TV runs Linux(Phillips TV), my Qlogic FC switches run Linux, I know F5 equipment runs on Linux, my phone runs Linux(Palm Pre). It really is pretty crazy how far Linux has come in the past 10 years. And I’m pretty convinced the GPL played a big part, making it more difficult to fork it off and keep the changes for yourself. A lot of momentum built up in Linux and companies and everyone just flocked to it. I do recall early F5 load balancers used BSDI, but switched over to Linux (didn’t the company behind BSDI go out of business earlier this decade? or maybe they got bought I forget). Seems Linux is everywhere and in most cases you never notice it. The only way I knew it was in my TV is because of the instructions came with all sorts of GPL disclosures.

In theory the BSD licensing scheme should make the *BSDs much more attractive, but for the most part *BSD has not been able to keep pace with Linux(outside some specific niches I do love OpenBSD‘s pf) so never really got anywhere close to the critical mass Linux has.

Of course now someone will tell me some big fancy device that runs BSD that is in every data center, every household and I don’t know it’s there! If I recall right I do remember that Juniper’s JunOS is based on FreeBSD? And I think Force10 uses NetBSD.

Also recall being told by some EMC consultants back in 2004/2005 that the EMC Symmetrix ran Linux too, I do remember the Clariions of the time(at least, maybe still) ran Windows(probably because EMC bought the company that made that product rather than creating it themselves)

September 7, 2010

All I want is a DB9

Filed under: linux,Random Thought — Tags: , , , , , — Nate @ 10:25 pm

Ok maybe that’s not all I want, but it’s a good start.

I got a new laptop recently, a Toshiba Tecra A11, really nice laptop. A couple of jobs ago I had a Toshiba Tecra M5 and liked it a lot, it had a couple glitches with Linux but for the most part it worked well. The Tecra A11 by contrast, no glitches with Linux, at least not yet. I’ve been using it about three weeks now, everything from wireless, to audio, to 3D,  microphone(first time I’ve ever used a microphone in linux, first time in easily ten years I’ve used a microphone on a PC period), and even webcam worked. And most importantly, suspend/resume has been 100% reliable. Really nice to see. It is certified with Ubuntu 10.04 64-bit which is what I’m running.

But that’s not really what this post is about, I wasn’t expecting it, so didn’t look for it, but was overjoyed when I looked and saw that this brand new business laptop had a DB9 serial port, a REAL serial port! Woohoo! I mean my M5 had one too and that was great, I just thought Toshiba had jumped on the train of let’s get rid of serial ports.

What a sight to see. I mean what Linux/Unix/Network geek in their right mind can get by without a serial port? Yeah I know you’ve been able to get those piece of crap USB serial adapters for some time, but I’ll take a DB9 any day! Especially when my favorite network gear uses native DB9 on their stuff too.
(Sorry couldn’t resist getting some purple in there, not enough color on this blog)

I was a fan of the IBM Thinkpad T-series for the longest time, until Lenovo bought them, was introduced to Toshiba a few years ago and they are by far my favorite laptop. If it’s going to be my main machine for work, then it’s gotta be something good. The Tecra line is it, the new T series for me.

Laptop specs:

  • Intel® Coreâ„¢ i7-620M Processor 2.66 GHz (3.33 GHz with Turbo Boost Technology), 4MB Cache,
  • Genuine Windows® XP Professional, SP3 with Windows® 7 Professional Recovery Media,
  • 8GB DDR3 1066MHz SDRAM (4096MBx2)
  • 320GB HDD (7200rpm, Serial ATA),
  • Nvidia® NVSâ„¢ 2100M with 512MB DDR3
  • Keyboard without 10-key numeric pad (black)
  • 15.6″ Diagonal Widescreen HD+ (1600×900) TFT LCD display,
  • Dual Point pointing device (Accupoint + Touchpad) and Media Control Buttons
  • Integrated Webcam and Microphone
  • Bluetooth® Version 2.1 +EDR
  • Toshiba 4-Year On-Site Repair + 4th Year Extended Service Plan

Customized pretty good they built it special for me! Mainly the “non standard” but “reccomended” keyboard(and custom matte LCD I hate the reflective screens). At first I was kind of upset they only offered ground shipping, I would be willing to pay more for faster shipping, but turns out it wasn’t ground after all, and they shipped it directly from China. Once it shipped it got here in about 4 days I think, through Alaska, then somewhere out midwest at which point I thought it was going to be put on a truck and driven back to Seattle only to find it hopped on another plane and flew to me instead.

September 14, 2009

Fix hanging vmware tools on linux

Filed under: Virtualization — Tags: , — Nate @ 5:48 pm

I can’t be the only one who has come across this, back in early June I filed a support case with VMware around the fact that roughly 90% of the time when the latest version of vmware-tools that shipped with vSphere loaded on my CentOS 5 systems it would hang part way through, if I logged into the console I and just pressed <enter> it would continue loading. Naturally the Tier 1 support rep was fairly useless, wanting me to do some stupid things to get more debug information.

I went off on my own and traced down the problem to the vmware-config-tools.pl script towards the end of the script at around line 11,600, where it tries to make a symlink. If I disable the offending code the problem stops(the link it’s trying to create is in fact already there):

sub symlink_icudt38l {
my $libdir = db_get_answer('LIBDIR');
install_symlink($libdir . '/icu', $gRegistryDir . '/icu');
}

If your interested in the strace output:

[..]
[pid  7228] <... read resumed> "", 4096) = 0
[pid  7228] --- SIGCHLD (Child exited) @ 0 (0) ---
[pid  7228] fstat(4, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
[pid  7228] close(4)                    = 0
[pid  7228] rt_sigaction(SIGHUP, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid  7228] rt_sigaction(SIGINT, {SIG_IGN}, {SIG_DFL}, 8) =  0
[pid  7228] rt_sigaction(SIGQUIT, {SIG_IGN},  {SIG_DFL}, 8) = 0
[pid  7228] wait4(7244, [{WIFEXITED(s)  && WEXITSTATUS(s) == 0}], 0, NULL) = 7244
[pid   7228] rt_sigaction(SIGHUP, {SIG_IGN}, NULL, 8) = 0
[pid   7228] rt_sigaction(SIGINT, {SIG_DFL}, NULL, 8) = 0
[pid   7228] rt_sigaction(SIGQUIT, {SIG_DFL}, NULL, 8) = 0
[pid  7228] lstat("/etc/vmware-tools/icu", {st_mode=S_IFLNK|0777,  st_size=25, ...}) = 0
[pid  7228] read(0, 

The last line there the system is waiting for input, when I hit <enter> it continues loading.

The support case sat..and sat..and sat. Then a couple of weeks ago some manager called me up and wanted to know how the case was doing. I guess they didn’t spend any time on it at all. I told him I already found a workaround, and he said because of that they were going to work up a KB entry on it then close the case. Then another week passes and I get an email saying OH! We see you found a workaround, we’ll forward that to engineering and get back to you. Yes the workaround I sent on JUNE 16.

So hope this helps someone, I’ll update this when/if they get a KB entry out on it. It’s certainly saved me a lot of time, it is very annoying to have to connect to each and every system to press enter to get it to continue to boot to workaround this bug.

« Newer Posts

Powered by WordPress