How to convert an integer to little endian or big endian

As discussed in my previous post "Little endian vs Big endian", there are some scenario where we need to take care of endianness of system. Consider an example where you want to send data over network to some remote systems. The systems can be of any endianness. All interested parties decide that the data transmitted over internet will be in big endian format. The system receiving data will then convert it to local endian format.

Let's write function to write and read bytes from a 64 bit integer. I am considering 64 bit integer because an endian related C program written with int may work fine on 32 bit platform but will fail on 64 bit platform. So, we will be using strict data types. More on C data types related issues here.

1. Extract bytes in big endian sequence from uint64_t irrespective of the execution platform.

void
uint64ToByteArray (uint64_t num, size_t bytes, unsigned char *arr)
{
size_t i;
unsigned char ch;
for (i = 0; i < bytes; i  )
 {
   ch = (num >> ((i & 7) << 3)) & 0xFF;
   arr[bytes - i - 1] = ch;
 }
}

For UINT64_C (0xabcdef1234567890), we will get the following sequence of bytes irrespective of platform.
ab cd ef 12 34 56 78 90

Infact, we can make this function more generic by adding a type parameter which will specify the sequence in which we need the bytes. Lets support little and big endian.

#define LITTLE 0
#define BIG 1

void
uint64ToByteArray (uint64_t num, size_t bytes, unsigned char *arr, int type)
{
size_t i;
unsigned char ch;
for (i = 0; i < bytes; i  )
 {
   ch = (num >> ((i & 7) << 3)) & 0xFF;
   if (type == LITTLE)
 arr[i] = ch;
   else if (type == BIG)
 arr[bytes - i - 1] = ch;
 }
}

So, if type == 0, byte sequence will be 90 78 56 34 12 ef cd ab
and if type == 1, byte sequence will be ab cd ef 12 34 56 78 90

2. Reconstruct uint64_t from bytes in little/Big endian sequence irrespective of the execution platform.

uint64_t
byteArrayToUInt64 (unsigned char *arr, size_t bytes, int type)
{
uint64_t num = UINT64_C (0);
uint64_t tmp;

size_t i;
for (i = 0; i < bytes; i  )
 {
   tmp = UINT64_C (0);
   if (type == LITTLE)
     tmp = arr[i];
   else if (type == BIG)
     tmp = arr[bytes - i - 1];

 num |= (tmp << ((i & 7) << 3));
 }
return num;
}

So, if byte sequence is 90 78 56 34 12 ef cd ab and type == 0, then uint64_t will be 0xabcdef1234567890

Little endian vs Big endian

Endianness in simple words is ordering of bytes in memory to represent some data. Computer memory in general is visualized as a sequence of bytes. But, in medium or high level languages like C we work with data types with size more than a byte.

Consider a 32 bit integer (in hex): 0xabcdef12
It consists of 4 bytes: ab, cd, ef, and 12. Hence this integer will occupy 4 bytes in memory. Say we store it at memory address starting 1000. There are 24 different orderings possible to store these 4 bytes in 4 locations (1000 - 1003). 2 among these 24 possibilities are very popular. These are called as little endian and big endian.

• Little endian - Stores the Least significant byte at the lowest address. Example: Intel Pentium Processors.
• Big endian - Stores the Most (Big) significant byte at the lowest address. Example: Sun/SPARC, IBM/RISC 6000.
  

On little endian system, memory will be like:

Address	Value
1000	12
1001	ef
1002	cd
1003	ab

On Big endian system, memory will be like:

Address	Value
1000	ab
1001	cd
1002	ef
1003	12

Now, the good news is that usually we don't need to care about endianness. It's taken care by Hardware Platforms, and Compilers. But, in some scenarios we need to care about endianness. A common scenario is when the data need to be exchanged between different systems. In such a situation, a standard layout is specified. Example: network protocols like TCP use Network Byte Order which is big endian. Thus the writers have to ensure that the data they write is in standardized order.

Before learning how to write/read in different orders lets play around and take a simple problem to determine endian of a given system.

There are 2 easy ways.

1. Write a known integer value in a binary file and view the file contents using a hex utility.

This method is more visual but is easy to implement and easy to detect layouts other than little/big endian. (Yes, there are other endians too like mixed endian).

Lets dump a 64 bit integer to a file:

#include <stdio.h>
#include <inttypes.h>

int
main ()
{
uint64_t integerForTesting = UINT64_C (0xabcdef1234567890);
FILE *fp = NULL;
fp = fopen ("dump.bin", "wb");
if (fp)
{
   printf ("Writing 64 bit number = 0x%" PRIx64 "  to dump.bin as it is\n", integerForTesting);
 fwrite (integerForTesting, sizeof (uint64_t), 1, fp);
 fclose (fp);
 printf ("Done writing\n");
 return 0;
}
printf ("Not written\n");
return 0;
}

Now, dump the contents of binary file (dump.bin) using command line hex utility like od.
#od -t x1 --width=8 dump.bin

If it dumps something like
0000000 90 78 56 34 12 ef cd ab
0000010
then the system is little endian.

If it dumps something like
0000000 ab cd ef 12 34 56 78 90
0000010
then the system is big endian.

2. Programmatically by looking at the value of byte stored at the starting address of a known number.

This method is limited in scope as it looks at only the first byte and only on the basis of that distinguishes between big and little endian. It won't detect other endian systems.

#include <stdio.h>
#include <inttypes.h>

int
main ()
{
uint64_t integerForTesting = UINT64_C (0xabcdef1234567890);
unsigned char *ch = (unsigned char *) &integerForTesting;
FILE *fp = NULL;
if (*ch == 0x90)
{
 printf ("this is little endian system\n");
}
else
{
 printf ("this is big endian system\n");
}
return 0;
}

C - Integer constant out of range warning

C doesn't define sizes exactly. C types such as int and long are defined as being "at least 16 bits'' and "at least 32 bits'' respectively.

If we want to define variables which have a specific size independent of platform, then we can use types such as int32_t, int64_t defined in sys/types.h header file.

So we would assume that the following code should work:

#include <stdio.h>
#include <sys/types.h>

int64_t num;
num = 0xabcdef1234567890;
printf("num = %llx\n", num);


Intended O/P: abcdef1234567890

BUT on some platform/compiler combination the output can come out to be 34567890.

Such as on IBM/AIX, compiler will give a warning

1506-207 (W) Integer constant <value> out of range.

Whereas GCC/LINUX will print the intended output.

The reason is that the integer literal 0xabcdef1234567890 wasn't explicitly specified to be of type int64_t.

A solution to this is to specify literal as 0xabcdef1234567890LL but a better way of doing this is to use macros in stdint.h (included in inttypes.h)

#include <stdio.h>
#include <inttypes.h>

int64_t num;
num = INT64_C(0xabcdef1234567890);
printf("num = %" PRIx64 "\n", num);

INT64_C will itself append "LL" or appropriate size specifier to literal depending on platform.