Tag Archives: compiler

When LD_PRELOAD doesn’t work

So, I recently had need to patch a binary. Due to some poor recording, I couldn’t rebuild the binary from source as I didn’t know what version of the source it was, and this was quite important. It looked like the source had been patched with one or more cherrypicked updates, and recompiled.

So, I was left with a quandary. I needed to change one instance of strcmp to strcasecmp, without breaking everything else and losing the exact patch(es) that had already been applied.

In the end I decided to use LD_PRELOAD and write a replacement for strcmp. It was only going to need to change on one specific search term, and so I could easily test the values passed in (using strcmp ironically) and replace strcmp with strcasecmp in this one specific instance.

However, after writing it, and LD_PRELOADing it, it just didn’t work. The function was there, because I could see debug coming out of it when other shared objects accessed strcmp, but the main executable itself, it just wasn’t having it.

A bit of checking, and some ltrace to prove it, and I found that when you use gcc with optimisation turned on, it inlines strcmp.

Now I know that strcmp is in a library so technically can’t be inlined, but gcc turns out to have its own version of various simple library functions, and substitutes them for the libc versions.

-fno-builtin as a gcc option stops this happening. But without that, you can’t use LD_PRELOAD to affect strcmp. Or several other functions.

Logging with a bit more brain

I expect many of you will use the preprocessor macros of __LINE__ and __FILE__ for logging. It’s fairly standard practice, but if you don’t, then I’ll introduce you to it.

#define LOG(x...) log(__FILE__,__LINE__,x)

This will allow you to do, for example

LOG("Something went wrong\n");

and it will include the file and the line number in the log function as the first two parameters.

That’s fairly common. Now, what’s less common is a thing I worked out for more complex logging.
I needed to log a message which may appear very regularly, but I didn’t want to spam logfiles with millions of them, I only needed to see the first one. I thought that I’ll have to tag the log with some kind of hash value for checking when it was last used, or something like that, when I had a brainstorm. It could all be done in the preprocessor.

#define LOGO(x...) { static int loginst__LINE__=0; if (loginst__LINE__==0){ loginst__LINE__=1; log(__FILE__,__LINE__,x);}}

As you can see, the variable loginst_nnn will only be used once per file, so is unique (unless you use LOGO twice on one line, don’t do that. The macro definition has to be all on one line, obviously. I mention this because wordpress is likely to wrap it unless your screen is very wide.

That got me thinking, what else can I do with the preprocessor like this. And I came up with:

#define LOGN(n,x...) { static int loginst__LINE__=0; if (loginst__LINE__++==0){ log(__FILE__,__LINE__,x);} if (loginst__LINE__==w) loginst__LINE__=0;}

This one logs every nth occurence of the log message. I also came up with

#define LOGT(t,x...) { static time_t loginst__LINE__=0; if (loginst__LINE__<=time(NULL)){ log(__FILE__,__LINE__,x);loginst__LINE__=time(NULL)+t;}}

which only logs at most once every t seconds.

All in all, I was quite pleased with that. They’re simple, effective, and a LOT more efficient than some other solutions I’ve seen out there.

 

Profiling of basic glibc functions

There’s a lot of anecdotal evidence out there about what to do and what not to do. Standard wisdom says that malloc is expensive, for example, and realloc is even worse. But for all of the times I’ve seen people say this, I haven’t seen any concrete numbers.

So, let’s make some.

I intend to keep this post updated as a growing list of functions from glibc, to show how they compare to each other, and how efficient they are.

Method:
Loop a million times running each function once, and analysed using kcachegrind

Results

sprintf (100 bytes, 5 variables)     1,752
pthread_create                       1,115 
fclose                                 575
fopen                                  527
pthread_detach                         405
malloc                                 101
free                                    84
pthread_mutex_lock (always available)   31
pthread_mutex_init                      31
pthread_mutex_unlock                    29
gettimeofday                            12
pthread_mutex_destroy                   10
time                                     8
close                                    7
open                                     7

Getting gcc to warn you when you mess up stdargs

This article was originally published on the LGP Blog on Wednesday, January 20th, 2010

Sometimes, you may write functions in C that do things in the same way as printf, using stdargs.

An example of this would be something like this short debug function

int ptf(const char *fmt,...)
{
  va_list varlist;
  FILE *fp=fopen("/tmp/debug","a");
  va_start(varlist,fmt);
  vfprintf(fp,fmt,varlist);
  va_end(varlist);
  fclose(fp);
}

This function isn’t rocket science, it just simply appends your string into a file. It is a simple time saver utility.

However, using it can be a problem. You can do something like this

int x=1;
ptf("Error %s\n",x);

And gcc will say ’sure, no problem’.

But running the code will always crash. It tries to interpret the integer as a string.

This is the kind of thing that should be picked up on by the compiler. And in fact it can be, quite easily.

In your prototype for the function, you would have something like

extern int ptf(const char *,...);

This is pretty standard, and no surprises there. However, gcc has the capability to be given a hint as to how this function should be handled. You can instead prototype the function using

extern int ptf(const char *,...) __attribute__ ((format (printf, 1, 2)));

This tells gcc to treat the parameters 1 and 2 as the parameters to printf (which it knows how to check for errors). It will then check parameter 1 (the format string) against what is passed in starting at parameter 2 (the …). If an incorrect data type is used, this will now be detected and flagged up as a warning, in exactly the same way as an incorrect type used in a printf.

Some gotchyas in porting from Visual C++ to g++

This article was originally published on the LGP Blog on Thursday, December 3rd, 2009

Todays technical article is going to go into a couple of issues that we have run across porting from Visual C++ to g++.

I will say right now that I am an application developer, I don’t know the fine details of how the inner workings of the compilers work, so some of this is merely an educated guess as to WHY the problem happens, but the effect and solution is correct.

The first issue for today is a fairly rare situation where g++, upon hitting a certain piece of code that builds fine under Visual C++, gives the helpful error message:

some/source/file.cpp:172: sorry, unimplemented: called from here

Now, I don’t know about you, but I find that error message singularly unhelpful. It took some time to run this one down the first time. It seems  that g++ and VC++ perform some symbol resolution in different passes to each other. Because what this message is saying is that ‘at line 172, you have tried to use __forceinline, or something you reference has a member function that has __forceinline in it, but I do not know what the thing is, so I cannot inline it’.

EDIT: g++ does not have the __forceinline keyword. There is an equivalent which does the same job however

#define __forceinline inline __attribute__((always_inline))

As VC++ has no problem with the same calls, the only real answer must be a compiler difference that allows VC++ to get away with something that g++ doesn’t. The simple solution is to just change __forceinline to inline and let the compiler inline as it sees fit. Otherwise, you will need to hunt down the exact problem and resolve the order of events for g++

The second issue I thought I would raise today is one that actually happens quite commonly on porting from Windows to Linux. This is the case of bad memory management.

On Windows, the way memory is allocated allows for a certain sloppiness in code. While HIGHLY unrecommended, this is the sort of thing that happens in Windows code all the time. The following is a highly simplified example of the problem.

char *str;
str=(char *)malloc(6)
strcpy(str,"hello");
free(str)
switch (str[0])
{
  ...
}

Now, as you can see here, the application uses the variable str right after it has been free()’d. On Windows it seems that this kind of thing can be gotten away with. The memory manager on Windows is highly unlikely to either assign this memory address somewhere else, or to mark it as unavailable. On Linux however, you will often find that this kind of code leads immediately to a segmentation fault.

The example above is highly simplified, but illustrates what some of the more complex segmentation faults we have seen boil down to.  If you see an error like this, which works on Windows and not on Linux, check your order of accessing. I know it sounds obvious, but it happens so often in commercial code that I feel it is worth stressing – free memory only when you have REALLY finished with it.