sstream of consciousness - Software engineering

Confluences, filesystems and such...

Sun, 15 Oct 2006 21:03:32 -0600

I added two confluences to my list. Victor and I spent three days in the beginning of September in Northern Ontario and we logged two successful visits. You may read the full trip reports 50 N 85 W and 49 N 83 W. We are planning more confluence visits, some of them may involve quite a bit of snowshoeing.

On unrelated note, I'm starting to worry about future of some filesystems. Companies behind two brilliant filesystems - XFS and ReiserFS are going through hard times now. Hans Reiser - founder of Namesys and creator of namesake filesystem is arrested on suspicion of murder. SGI - sponsor of XFS - filed for bankruptcy.

Unfortunately, there is not much going in filesystem world lately. Subject area is actually extremely complex and very conservative (it is your data at stake after all). Microsoft took WinFS out of Vista and the only notable advancement I can think of is ZFS by Sun - not exactly new but it is getting a lot of traction lately.

One of the most interesting reads lately were answers of several famous programmers to a Polish student's questions, published on student's blog. Torvalds, Stroustrup, Norvig, Bray, Gosling and others share their thoughts on next big things, best books, most useful skills forprogrammers, and many other topics.

On day-to-day Software Development

Wed, 13 Sep 2006 10:16:36 -0600

GNU linker on Fedora Core 5 is horribly slow on moderately large C++ projects with debug information. Apparently, this is a known problem. This is frustrating to wait more than one minute for a linkage of a project that normally links n a matter of seconds. Tried to run yum update but that did not solve the problem - and package download was very slow. I watch this PR and wait for the patch.

New glibc 2.4 now has a detection of mulitply freed memory. While this is not a full-blown Purify or my favourite Valgrind but this introduces almost no overhead in runtime and is constantly on. Nice work!

Shooting the troubles

Wed, 08 Feb 2006 21:30:35 -0700

Probably the most interesting part of web log statistics for my site is a page with keywords that people use in search engines to come to my pages. The most popular search is for information about C++ string streams. Ironically, this blog does not provide much help here, though I put some useful links on my front page.

Over the course of my career I bumped into many different technical problems. Often these problems required a lot of time or mental effort to investigate. Needless to say, web search engine made that task way easier, but sill there were problems I had to spend considerable time on. Once I had this blog I have started to put some results of my troubleshooting sessions, primarily as a sign of gratitude to all the people who do the same.

So, when I look at search term statistics, I am always glad to find that people came to the solution at my site, using search keywords describing their problems, like the problem of putting spaces to LD_PRELOAD environment variable or Coloured bash prompt screwing multiline input in a terminal.

Maybe separate articles or pages instead of blog entries is more appropriate for that kind of texts, but I am pretty happy with the present format now.

In near future I am going to post a couple of entries explaining problems related to GNU C library application binary interface (ABI) revisions and incompatibilities and problems with C++ language ABI.

How to specify library name with spaces in LD_PRELOAD

Mon, 19 Sep 2005 15:30:45 -0600

Recently several of my colleagues bumped into an interesting problem.

We use LD_PRELOAD environment variable in couple of places. For those of you not familiar with it: LD_PRELOAD is an environment variable which allows you to specify dynamic libraries which will be loaded before all other libraries. This technique allows you to intercept calls to standard libraries is used by many debugging and analysis tools. Or you could alter behaviour of tsndard libraries for different purposes (e.g. fault injection).

As other variables like PATH or LD_LIBRARY_PATH, this variable may contain list of library namesseparated by colons. But... for compatibility with legacy systems it is possible to separate LD_PRELOAD elements by spaces. And older systems did not understand escaping so it turns oout it is impossible to put full library paths into LD_PRELOAD if they contain spaces.

But here is the trick. You may put library directory name(s) with spaces into LD_LIBRARY_PATH (which is kind of a PATH but for dynamically linked/loaded libraries) and put short library name(s) into LD_PRELOAD. Of course, short library file name must not contain spaces but this is less of a limitation.

My submission to Slashdot got accepted.

Mon, 30 May 2005 22:21:15 -0600

Now that's kinda cool!!

Slashdot editors accepted my submission about Ulrich Drepper's post that I blogged about recently. It spawned fierce and interesting discussion.

Many slashdotters point out that bugs showing up on specific platforms are hidden and hard to debug problems. That is why having code running on many platforms is a virtue. That is completely correct: I discovered several many memory-related bugs just because my code broke on Solaris (whereas it worked seemingly fine on Linux).

Another valid point: if you have your code ported to many platforms you are better prepared to face new platforms rising in popularity, like MacOS X nowadays. Also, if developers care about portability, code usually tends to be more modular and clean. Funny enough, reverse is also true, one of slashdot readers pointed out that InfoZip's code is a complete mess due to the fact that it supports so many platforms.

However, I believe Ulrich was not talking about this kind of bugs. At least this was not a main focus. I presume main complaint was about platform differences that are so hard to track. Starting from binary code, up to different kernels, up to different firmware, up to different peripheral hardware, up to varying implementations of C library and other interfaces like POSIX, up to different set of system utilities etc.

Many efforts (finished and ongoing) was made to ensure at least an existance of some greatest common denominator (POSIX, standard C library, LSB etc), but situation is still far from ideal and unlikely will ever be.

Supporting of non mainstream platforms considered harmful?

Sun, 29 May 2005 10:44:00 -0600

Ulrich Drepper, RedHat employee, wrote 2 extremely interesting posts: first, titled "Users and complexity vs Bugs", and second tilted "Dictatorship of minorities". To sum it up: these articles are about difficulties of cross-platfrom development. The latter post is especially interesting. To put it simply, Ulrich argues that it is unnecessary to support as many different platfroms as open source projects strive to. Moreover, it is even unfair to support proprietary platforms in free software.

documentation

Sat, 28 May 2005 16:36:53 -0600

A lot of people come to my webiste from Google and Yahoo. Some of them found my site using keywords related to C++ standard library header . Maybe they are just curious, but my site may not be what they want. For those people I created this page that contains some links to the actual documentation of C++ <sstream> header.

* MSDN page
* Apple Developer or GNU Standard C++ library.
* Overview of iostream classes

Hope this helps.

Why Microsoft compilers decorate variable names

Wed, 13 Apr 2005 10:07:39 -0600

If you are a seasoned system programmer, it is very likely that you had to write or maintain mixed C/C++ application. And if you had, then you know that if you want to make functions in the translation unit compiled from C source available for C++ code, you must delcare them as extern "C". The same applies to functions implemented in C++, which you want to make available to C programs. Usually this is is done with preporcessor in that way (YMMV though):
#ifdef __cplusplus #define EXTERN_C extern "C" #else #define EXTERN_C extern #endif EXTERN_C void c_style_function(void);

You need extern "C" in C++, because otherwise C++ compiler will mangle function symbol, so it will be iaccessible from code compiled from C. The rational behind C++ mangling is that you may overload functions (to put it simply, functions with the same name but different parameter lists may have different implementation).

However I was very surprised when I found that Microsoft C/C++ compiler mangles data symbols. Back then I though: "Why on Earth, they do that? You cannot overload variables!".
read more

Essential books for Software Engineering/Computer Science student

Sun, 10 Apr 2005 16:32:06 -0600

Some time ago I came across Josh Kaufmann's Personal MBA progam which is basically a list of books and blogs that are kind of a substitute of an MBA program, and you don't have to spent so much money on it.

Inspired by this list I decided to gather a list of most important books in Software Engineering and Computer Science. Almost all of these books I either read at whole or heavily used as a reference material. I studied mathematics with books by russian authors, so I tried to find kind of a substitute book for each subject. Many books in this list are "THE" books on its subject matter, "classic" I would say.
read more

Defects in the Linux Kernel found by Klocwork defect detection tools

Thu, 17 Mar 2005 12:50:41 -0700

I am working on improving Klocwork's defect detection tools and. During that process we use large open sourced codebases to test our performance, false positive rate etc. One of these codebases is a Linux kernel.

Recently I posted a couple of potential defects to Linux kerenel mailing list, and OSDL developer Randy Dunlap acknowledged that these are real defects, impact is very limited though. You have to have sysadmin capabilities in order to have a possibility to crash the application.

However that was just a "foot in the door". Expect tread more

More Microsoft API weirdness.

Wed, 26 Jan 2005 09:32:15 -0700

Again, wandering through MSDN, I found interesting peculiarity of Microsoft's implementation of standart C functions.

There are two functions, wsprintf and swprintf, that do pretty much the same - they perform formatted output of wide-character text to a string buffer. So, what is the difference, beside permutated letters?
read more

What Microsoft documentation will not tell you

Wed, 26 Jan 2005 07:58:40 -0700

Recently I was exploring functions from WinAPI, namely shell path handling functions, like PathCombine, and PathAppend. There is a limit on path length called MAX_PATH and destination buffers for that functions must be at least MAX_PATH+1 characters in size (which is not the same as MAX_PATH bytes, by the way!). Let's take for example PathAppend. Basically, this function concatenates two paths, inserting backslash if necessary. So what if lengths of two arguments are greater than MAX_PATH in total? I would expect that destination buffer would contain some trimmed version of concatenated path But this is not so! Destination buffer will have zero length.
read more

Plugin infrastructure on Windows

Mon, 22 Nov 2004 15:18:44 -0700

I was busy building a plugin infrastructure. Here is a couple of tricks that may be useful for other developers.

Plugin is basically a dynamically loaded library, loaded by main application at runtime. Then application gets and address of a library entry function. Library entry function may do whatever is needed but at the end of the day it should register callbacks - functions provided by extension(s) and hooked onto specific extension points. These are ABCs of plugin programmming.

Now, the problem is that dynamically loaded library needs to use functions from the main application.

On Unix, .so file was compiled without problems but 'dlopen' function failed at runtime. Solution was simple: I just had to link main application with gcc's '-export-dynamic' option. That made all functions in the main application exposed for dynamic linking.
read more