Best of IRC: Introducing glibc 2.2

This past week we had a discussion on IRC about glibc 2.2. There are probably still a lot of people who don't know what glibc is. So, before we get into the discussion about what's new in 2.2, let's first cover a few things about glibc.

Background

Glibc stands for GNU C Library. Glibc provides the core functionality for C programs to run (in other words, just about every program on your system will use glibc). Glibc 2.2 represents the latest release of the glibc 2 tree originally introduced in January 1997 when 2.0.28 was stable 2.1.22 was bleeding edge. Since glibc replaces the libc4 and libc5 that came before it, it is occasionally named libc6 - I'll show you what I mean by that in just a second.

So how do I know which version of libc I'm using?

If you look in /lib you'll see all the libraries needed for system bootup. The names of these libraries follow a patterm of libname.so.major.minor. You'll probably find a few in there that simply end in a major number, or simply end with .so. The reason for this is simple. It would be quite a pain to recompile programs each time the version number changed so instead programs reference libraries by these shorter names and depend on the dynamic linker to make these shorter names symlinks to the current version, hence the reason for running ldconfig after installing new libraries.

Getting back to glibc, we should see the libc libraries in /lib. The one to look for is libc.so.6; don't worry if you still have libc.so.5, it's only there for compatibility. Notice that libc.so.6 is a symlink, if we look at it with ls -l we can see which version of glibc we're using:

mbm@reboot:~$ ls -l /lib/libc.so.6
lrwxrwxrwx 1 root root 13 Jan 15 00:31 /lib/libc.so.6 -> libc-2.2.1.so*

So what's new in 2.2?

For that we turn to the IRC discussion given by David Ford, and moderated by Derrick Lewis (Blu3 and dlewis respectively).

>>> topic/#Linux.com-live! by dlewis -> Linux.com-Live! Event!!!! How to obtain, compile and install the GNU C Library: glibc 2.2
<dlewis> ok
<dlewis> we will begin.
<Alpha_Zulu> yeah!
<dlewis> Hello everyone, and welcome to another exciting Linux.com-Live! EVENT!!!!
<dlewis> Today, we are in for a treat. The Uber-Geek of Linux.com, David Ford, is with us.
<Alpha_Zulu> hi Mr. Ford
<dlewis> David is Mail/Systems Administrator for Linux.com. Outside of Linux.com David is a computer security consultant, a paid hacker (the good kind).
<dlewis> David will be discussing how to obtain, compile and install the GNU C Library: glibc 2.2.
<DFx> My dream job (good hacker)
<lucretia> he got my 85% packets then
<Blu3> =)
<dlewis> After the discussion, we will have a Q&A Period.
<dlewis> I would like to ask you to forward ALL questions to me and I will then bring them forth to David.
<dlewis> And that you do not speak while David is speaking.
<dlewis> I will now turn the floor over to David.
<Blu3> g'evening people
<Blu3> thank you derrick
<Blu3> last time i gave a lecture here, i went on and on
<Blu3> a few people asked me to make it a bit shorter ;)
<Blu3> so, this evening's talk will probably be less than 30 minutes altogether
<Blu3> let's start off with a quick description and introduction to GNU GLIBC
<Blu3> GLIBC is as the acronym expands, GNU C library
<Blu3> don't ask why it comes out that way, it's like UTC, universal coordinated time :)
<Blu3> glibc as i'll refer to it for this evening is meant in context of installing and using on a Linux system
<Blu3> it certainly isn't limited to Linux but this discussion will assume that you are building, installing, and using it on linux
<Blu3> most of your programs are linked to this library and require it to operate
<Blu3> unless it exists properly then your system is likely not functional
<Blu3> the main web site for glibc is at http://www.gnu.org/software/glibc/<Blu3> it's downloadable at ftp://ftp.gnu.org/pub/gnu
<Blu3> now before discussing how to compile and install it, i'll cover some of the new parts of the library
<Blu3> a quick synopsis is:
<Blu3> runtime support for pointer bounds checking
<Blu3> there is support in CVS gcc for bounds checking against pointers
<Blu3> when this becomes fully integrated in both gcc and glibc, programs can have pure protection against bounds overflow with pointers
<Blu3> i.e. slashing most exploits out of existence
<Blu3> even if there is a bug :)
<Blu3> - secure mode for nscd
<Blu3> nscd is the name services helper daemon
<Blu3> it assists the resolver in looking up hostnames and IPs, the like
<Blu3> BIND which is what the internet's DNS system runs on has been partially incorporated in this
<Blu3> BIND supports crypto communique between systems now and the secure mode for nscd is a step in that direction
<Blu3> - berkeley db libraries removed
<Blu3> due to the nature of BDB, this is now an aside, you may or may not include it, but it comes as a separate package now
<Blu3> - resolver code updated from 8.2.3-t5b
<Blu3> this helps speed things up and fixes some long standing problems. there is also a side effect here and i'll cover that in my 'gotchas' section
<Blu3> - k&r compat removed
<Blu3> K&R (kernigan and ritchie) compatibility has been removed. you need an ISO C98 or C99 compiler (current gcc) to compile glibc now
<Blu3> - rewrite of localedef to support multibyte charsets
<Blu3> the library is getting considerable more support for other languages, particularly those that are multi-byte
<Blu3> the last main new feature is --enable-kernel=X.Y.Z
<Blu3> this is used in two ways, for ix86 (386, 586...), it can be used to remove old code used only for legacy support.
<Blu3> for example, removing special support for a 1.x or 2.0 family kernel
<Blu3> and the second half of the answer leads into the gotchas
<Blu3> you -must- use this configure option when compiling for ia64 (alphas)
<Blu3> to compile on ia64, you need linux kernel at least 2.4.0-test4-000728 or later
<Blu3> in complement to this, ia64 glibc does not support debugging or profiling
<Blu3> covering the gotcha i mentioned earlier with bind, '_res' is now thread-local instead of global
<Blu3> some programmers may have relied on this variable to be global. if you are a program, take note that this is now a local context variable
<Blu3> utmpd is removed (utmpd was a daemon that watched and helped update your utmp file)
<Blu3> another very important issue is ipv6
<Blu3> much of the IPV6 code was updated, some rewritten altogether
<Blu3> the ABI (application binary interface) changed. what this means is programs that used ipv6 compiled against an older version of the library will no longer work as expected

Compiling and Installing

<Blu3> ok, compiling and installing
<Blu3> first, download the gcc source, you also need the linuxthreads tar
<Blu3> you do not need the crypt addon anymore, with the relaxation of export controls (USA), this can now be included
<Blu3> after untarring glibc, untar the linux threads tar inside the freshly untarred glibc directory
<Blu3> if you want to modify the default values, i.e. you are making a distro or really want the new glibc installed in /lib etc, then edit a file called 'configparms' in the glibc directory
<Blu3> here's an example:
<Blu3> cat > configparms >> "EOF"
<Blu3> # Begin configparms
<Blu3> slibdir=/lib
<Blu3> sysconfdir=/etc
<Blu3> # End configparms
<Blu3> EOF
<Blu3> the slibdir is the 'system library directory', or where the library modules will be installed
<Blu3> the sysconfdir is where the system config files goe that pertain to glibc
<Blu3> here are the options i usually use when i compile.
<Blu3> ./configure --prefix=/usr --exec-prefix= --enable-add-ons=linuxthreads
<Blu3> note carefully the blank exec prefix
<Blu3> if you put =/, then all the paths will end up like //lib and //etc when it attempts to find something
<Blu3> it won't break anything, it's just a matter of aesthetics
<Blu3> if your Linux kernel headers are in a different location, specify --with-headers=...
<Blu3> naturally ... replaces the path to the headers
<Blu3> which btw should point to the TOP level of the linux kernel source
<Blu3> now that you've gotten the ./configure done, simply type 'make'
<Blu3> and depending on your hardware, come back in about an hour
<Blu3> there are a few requirements to compiling and installing i need to mention
<Blu3> unless specified, these are all GNU tools
<Blu3> gettext, gperf
<Blu3> (optional, only needed for tests with make) perl
<Blu3> about 400M of space
<Blu3> current Linux kernel headers
<Blu3> and here is an important point a lot of people overlook.
<Blu3> the /usr/include/linux and /usr/include/asm links should and must point to real kernel source. the /usr/include/net and scsi however should not. those headers are now in glibc
<Blu3> it is also important to bear in mind that you should keep the kernel headers you compiled glibc against
<Blu3> i.e. if you compiled glibc with 2.4.0 headers, then you should keep those headers attached to glibc. instead of sym links, make them real directories and copy the headers those links pointed to
<Blu3> otherwise you may encounter bugs when writing software. they aren't really bugs but simple changes in APIs
<Blu3> this is however very rare but is worth remembering
<Blu3> now that things are all compiled you only need to install
<Blu3> there is one VERY important thing to remember here
<Blu3> do not overwrite your RUNNING libc
<Blu3> i.e. if you already have /lib/libc.1.2.3.4 installed and have compiled a new one, don't blindly copy or move the new one on top
<Blu3> you must rename the old one to a different version then run ldconfig
<Blu3> -then- and only then can you put the new compile in place
<Blu3> again, this is only for the RUNNING library. if the libraries are different versions (different file names) then you need not worry.
<Blu3> you should close your applications or at least save your work
<Blu3> i rarely have problems when uprading libraries on a running system but it is possible for strange things to happen
<Blu3> if they do, simply close programs as best possible and reboot in the unlikely event that programs continue acting odd
<Blu3> 'make install' is all you need
<Blu3> do keep in mind that glibc 2.2 takes about 35 megs when you install all the libraries, documentation, and locale data files
<Blu3> that's the end of the lecture, we'll now have a question and answer period. PLEASE send your questions to dlewis
<Blu3> he will combine them neatly and feed them to me
<dlewis> ok

Question & Answer

<dlewis> Please send ANY question to me...
<dlewis> just keep them on topic :)
<dlewis> I have received 1 question so far.
<dlewis> one of our audience members asked this:
<dlewis> "For those of us who are just playing around in Linux to get a feel for it, and use it for web development and basic beginner coding, is there any MAJOR reason why we should upgrade? I understand that a lot of the new features are important to major sys admins, but what about the newbie, anything worth upgrading for them?"
<Blu3> answer: yes. security. in particular i understand there is a bug just found today and a new version of glibc will be out asap
<Blu3> this bug is a local exploit where anyone can read any file, possibly leading to unauthorized shell access or otherwise
<DFx> i was reading something about that on bugtraq, perhaps its a good idea to upgrade then
<Blu3> yes
<Blu3> it's also a good idea to have current programs and libraries even if you're getting your feet wet
<Blu3> it's better to start off with something 'more correct'

<mike_d> i've heard that the new version of glibc has problems with c++ is this true?
<Blu3> no, that is gcc distributed with redhat
<Blu3> the versions of glibc and gcc that redhat 7.0 shipped with are between ABI and API specifications
<Blu3> thus c++ programs compiled on rh7.0 are unlikely to run on either previous or future versions
<Blu3> in light of this:
<Blu3> we recommend that _ALL_ redhat7 users upgrade to 2.2 (an upgrade RPM is available fromtheir website). it fixes the binary incompatiblity stuff redhat 7 had.
<Blu3> redhat recommends everybody upgrade immediately
<Blu3> thank you muks for reminding me of that
<Blu3> now programs compiled still won't run back with older glibc, but older binaries will run
<muks> :)
<Blu3> the GCC steering committee still recommends people only use official release versions
<Blu3> more questions?

<dlewis> Blu3: For the newbie, is it worth it to upgrade from 2.0-2.2?
<Blu3> yes
<Blu3> there are some small but measurable performance gains not to mention numerous bugs that were fixed
<Blu3> for those of you who are into serious programming, I omitted the in depth discussion of programming because we have another event happening shortly that gets into this
<dlewis> ok

<dlewis> next question:
<dlewis> As you said before, '_res' is no longer global but local. Is there any specific applications/programs that does this as global that you can name?
<Blu3> at present i am not aware of any
<dlewis> ok
<Blu3> it should only concern a very small number of programmers if any
<dlewis> Are there anymore questions for David?

<dlewis> Question:
<dlewis> Should all programs compile with this newer version of glibc, or will there be some incompatibilities?
<Blu3> by my understanding, unless you have peculiar quirks in your code, everything should compile just fine
<Blu3> if you are involved in ipv6, you should take a look at the function calls and make sure your expected input/output for the calls is the same
<dlewis> ok

<dlewis> Any more questions?
<dlewis> If not, then I would like to end this enlightening Live! Event!!!
<dlewis> I would like to thank the Uber-Geek, David Ford, for being with us.
<Blu3> i suspect a few people got the wrong start time :S
<Blu3> i'm not really that uber of a geek tho :)
<dlewis> It's always a pleasure to hear your knowledge.
<DFx> thankx mr. Ford
<Blu3> 'tis a pleasure to relevate
<dlewis> This entire Live! event will be recorded and put on the Linux.Com/Live website for your studying.
<dlewis> David will be around for questions you might think of.
<Blu3> yep
<Blu3> are any of you programmers?
<Daedron> yep!
<hugenerd> sort of :) not very good.
<mike_d> um aspiring
<DFx> im just starting to learn C+, i know perl for the use of cgi scripts, and thats about it for me.
<Maverick}> i'm learning C
<hugenerd> so I know I missed it, but was there any main point that I should know about?
<Blu3> good, good
<DFx> hugenerd; i can send you a log if you like...
<hugenerd> DFx: yes please.
<Blu3> hugenerd, there are a few
<Blu3> ok
<DFx> lemme dig it up
<Blu3> writing software is like a dream in creation
<Daedron> hear hear!
<Blu3> :)
<mike_d> wish my dreams were better :)
<Blu3> true programming is an art
<hugenerd> Blu3: cant you give me the top 3? :)
<Blu3> hugenerd, i covered make/install, some of the new things, and some gotchas
<Blu3> one of the most exciting new things is the bounded pointer checking
<DFx> hugenerd: just lemme remove the personal lines from the log..
<hugenerd> Blu3: ok. I got it compiled and working. And most of my code has been recompiled on the new libs.
<Blu3> when this is fully written in gcc and glibc, bounds overflow by pointer will be very hard
<Blu3> good to hear
<Blu3> hugenerd, there is a new option to remove legacy code, --enable-kernel=X.Y.Z
<Daedron> Blu3: what is the performance hit on that bounds pointer checking? is this feature on for development, or part of main runtimes used by all apps?
<Blu3> if you specify 2.4.0 for example, glibc will not be compiled for any legacy support below kernel 2.4.0
<Blu3> daedron, i don't know yet, it isn't finished
<hugenerd> Blu3: does it make a difference for speed?
<Daedron> Blu3: ack, yes, i remember - you said that. silly me.
<Blu3> it basically changes every pointer into an internal structure of three pointers
<Daedron> ah, so it'd only be feasible for debugging then I would suppose?
<Blu3> huge, i suspect it will have an impact, probably small though
<Blu3> no, it'd be good for runtime. you'll get a page fault/bus error instead of getting exploited
<Daedron> hmm, i dunno. as a game developer, I'm very interested in every last bit of performance.
<hugenerd> ok. I have 2 real questions. is the ldconfig that comes with glibc-2.2 better than the one that comes with ld.so? and is there a debugger that works well with multi-threaded programs?
<Blu3> then i recommend compiling and testing, then remove all the fences and bounds
<cireland> Has any of the development for the StrongArm CPU made it into the source tree?
<Daedron> ok. does the bounds checking increase memory usage, too? i would think it would...
<Blu3> hugenerd, i haven't had an opportunity to compare the two, personally i have a small problem with the new one that i need to research more before i attach fault. it could be user error :)
<Blu3> daedron, yes, but only at a cost of two (void *) sets
<Blu3> iirc
<Blu3> cire, let me check real quick
<Daedron> has anyone been successful getting glibc2.2 to work by using the Mandrake RPMs from Cooker on an MDK7.2 system?
<Blu3> cire, there seem to be a list of cross compile issues atm
<cireland> I believe it :)
<Blu3> cire, but it looks like it's moving along
<cireland> Good to hear. I can't wait to try it
<Blu3> the FAQ and other files have a growing count of ARM mentions
<Blu3> there were libm updates in late October and numerous header updates in September
<Blu3> actually, September had a lot of new arm files go in
<Blu3> daedron, no idea.
<Daedron> ok ^,^ I tried once one my MDK7.1 box and killed it totally (which is why I'm running 7.2 now)
<mike_d> thank you Blu3 it was educational
<Daedron> well, looks like it's time to try rebooting my new 2.4 kernel. thanks for the lecture, Blu3