C++ allows you to annotate functions with the inline keyword. From what I understand, this provides a hint (but no obligation) to the compiler to inline the function, thereby avoiding the small function calling overhead.

I have some methods which are called so often that they really should be inlined. But inline-annotated functions need to be implemented in the header, so this makes the code less well-arranged. Also, I think that inlining is a compiler optimization that should happen transparently to the programmer where it makes sense.

So, do I have to annotate my functions with inline for inlining to happen, or does GCC figure this out without the annotation when I compile with -O3 or other appropriate optimization flags?

inline being just a suggestion to compiler is not true & is misleading.There are two possible effects of marking a function inline:

1. Substitution of function definition inline to where the function call was made &
2. Certain relaxations w.r.t One definition rule, allowing you to define functions in header files.

An compiler may or may not perform #1 but it has to abide to #2. So inline is not just a suggestion.There are some rules which will be applied once function is marked inline.

As a general guideline, do not mark your functions inline just for sake of optimizations. Most modern compilers will perform these optimizations on their own without your help. Mark your functions inline if you wish to include them in header files because it is the only correct way to include a function definition in header file without breaking the ODR.