pondělí 7. července 2014

Useful things (for code)

Throughout the time, I've been working on various projects, I've found out several really useful and important things that are helpful ... a lot.

For example, modifying memory allocation/delocation (in C/C++; to track memory); creating various files and their parsers (for constants, configuration files, or even simple languages); creating complex log systems (you will need it!); and so on.

Allow me, to show some ideas behind these concepts in this and maybe following posts.

Memory tracking

From my own experience, I know that understanding how memory works takes very, very long time. And even after you've mastered it, you will have memory leaks in your code.

Of course, we have tools like valgrind, but sometimes you would like something simple, to just print out objects you alone have allocated. To detect whether they cause memory leak or not. Simply said, putting out log, similar to this:

Memory dump:
Record 2 (Engine::Computing) Size: 1 Aligned at: 16 At: 006EA930
Record 1 (Engine::Constants) Size: 32 Aligned at: 16 At: 006EA9F0
Record 18 (Engine::Edge) Size: 12 Aligned at: 16 At: 06E64CF0
Record 19 (Engine::Edge) Size: 12 Aligned at: 16 At: 06E64E30
Record 16 (Engine::EuclideanHeuristics) Size: 4 Aligned at: 16 At: 06E647D0
Record 17 (Engine::Graph) Size: 24 Aligned at: 16 At: 06E649A0
Record 0 (Engine::Log) Size: 256 Aligned at: 16 At: 006EA5B0
Record 4 (Engine::Triangle) Size: 48 Aligned at: 16 At: 06E62EE0
Record 5 (Engine::Triangle) Size: 48 Aligned at: 16 At: 06E63140

To print out F.e. allocation ID, type, size of allocation (in bytes), memory alignment and address. This alone is enough to catch tons of leaks.

All we need for this is some sort of structure to hold each single record, like:

/// Structure for tracking memory
struct MemoryRecord
 /// Size of allocated memory in bytes
 size_t mSize;
 /// Memory alignment
 size_t mAlignment;

 /// ID of allocation
 size_t mMemoryAllocationId;

 /// Pointer to data
 void* mPointer;

This is a simple structure, which stores all the data we need for records, except for one last thing - string describing the type. For this purpose I use standard map, like:

/// Map holding memory string (generated) and memory record
std::map<std::string, MemoryRecord*> g_mMemoryMap;

Now, the actual string for this memory map is type name plus some unique sequence for each given allocation (separated with some sort of symbol - e.g. it can look like Engine::Triangle_4dea813f). Now, why map, why not just plain vector or list. The answer is simple - what if we would like to search inside it (count the number of allocated triangles) - this is where map has some advantages over vector or list.

Of course we don't expect some light speed in the end, still, memory tracking should be used in debug version (so you can also step through the application and that way track down the leaks).

All we need now is to create our own Allocate and Free procedures (or methods). For me I simply call malloc (actually aligned_malloc) and insert new record into memory map, on release I free the record from memory map. It is nothing hard. Here is an example:

/// Allocates memory with specified size in bytes aligned on specific boundaries
/// Memory size in bytes
/// Memory alignment boundaries
/// Pointer to allocated memory on success, NULL on failure
void* Engine::Allocate(size_t size, size_t alignment, const char* classname)
 if (alignment == 1)
  return malloc(size);

 if (!(alignment & (alignment - 1)) && alignment < sizeof(void*))
  alignment = sizeof(void*);

 void* allocated = _aligned_malloc(size, alignment);

 // Store memory record, set up all data
 MemoryRecord* m = (MemoryRecord*)malloc(sizeof(MemoryRecord));
 m->mSize = size;
 m->mAlignment = alignment;
 m->mMemoryAllocationId = g_mMemoryAllocations;
 m->mPointer = allocated;
 std::string name = std::string("SharedMalloc_") + std::string(classname) + std::string("_") + boost::lexical_cast<std::string>(allocated);
 g_mMemoryMap.insert(std::pair<std::string, MemoryRecord*>(name, m));

 // Return pointer
 return allocated;

/// Frees memory aligned on specific boundaries
/// Pointer to memory to be freed
void Engine::Free(void* ptr, const char* classname)
 // Remove pointer from memory map
 std::string name = std::string("SharedMalloc_") + std::string(classname) + std::string("_") + boost::lexical_cast<std::string>(ptr);
 auto it = g_mMemoryMap.find(name);
 if (it != g_mMemoryMap.end())

 // Free pointer

The last step is, connecting these Allocate and Free to the classes. For this I use a very simple macro, like:

#define MEMORY_TRACKED_NEW_DELETE(classname) \
 void* operator new(size_t size)\
 return Engine::Allocate(size, 16, classname); \
 void* operator new[](size_t size)\
 return Engine::Allocate(size, 16, classname); \
 void operator delete(void *ptr)\
 Engine::Free(ptr, classname); \
 void operator delete[](void *ptr)\
 Engine::Free(ptr, classname); \ 

So in the end, a class that gets its memory tracked can look like:

/// Example class that uses memory tracking
class Node

As you can see, the implementation is straightforward. Of course this-like classes are very useful, during all stages in software development.

That is all for now. I will think off which of the useful things will take place next time.

1 komentář:

  1. Hello.
    I saw your post on Gamedev.net about real time ray tracing. I want to try and build a RT-RT on OpenCL. You mentioned that you had a project started and many ideas on the subject. Please I would like to contact you and try to borrow some of your ideas. Please contact me back at