Internals of the U-boot: The Shell and the Linker List

Introduction

U-boot offers a “hush” shell similar to the Linux shell. But the U-boot shell is much simpler compared to a typical Linux shell. A Linux shell provides two kinds of commands – shell-built-ins (e.g.: type, cd, [ ) and external commands (e.g.: grep, vim). The shell-built-ins are commands which are part of the shell’s binary, whereas external commands are binaries residing on file system. Invoking a shell-built-in is easier compared to invoking an external command. When a shell-built-in is invoked, it can be executed by invoking the function which implements it. Executing an external command involves looking up for that requested program in known paths of file system, setting up and running the process, and finally its teardown.

All the commands of U-boot are analogous to shell-built-ins of Linux shell. All these commands are C functions, and they finally become a part of the U-boot binary when it is built. U-boot uses an elegant mechanism called “Linker Lists” to include these commands into its final binary. This mechanism provides a scalable way to add commands to the U-boot. Let’s first look into how a new command can be added to the U-boot shell.

This article tries to explore the linker list feature used in U-boot to bind commands as a part of the u-boot binary. The same principles are used in the driver model of U-boot to bind drivers with the u-boot binary. Through this approach, adding a command or a driver becomes easier. This improves the maintainability of the code.

Workflow to add a new command

The source code for all the commands of u-boot are located in the cmd/ directory. There is one file per command, and the file is typically named after the command. Let’s take a look at the source code of the simplest command “echo”, located in the file cmd/echo.c.

#include <common.h>

#include <command.h>

static int do_echo(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])

{

……

}

U_BOOT_CMD(

echo, CONFIG_SYS_MAXARGS, 1, do_echo,

"echo args to console",

"[args..]\n"

" - echo args to console; \\c suppresses newline"

);

The function “do_echo” implements the “echo” command. It is defined static to limit its scope to the current translation unit (i.e., the file echo.c). This prevents the function “do_echo” to be invoked directly. “do_echo” takes three argruments –

1. cmd_tbl_t cmdtp – A structure which binds the command name with the function which implements it, and among other things, it provides some information which helps the shell to run the command.

struct cmd_tbl_s {

char *name; /* Command Name */

int maxargs; /* maximum number of arguments */

int repeatable; /* autorepeat allowed? */

/* Implementation function */

int (*cmd)(struct cmd_tbl_s *, int, int, char * const []);

char *usage; /* Usage message (short) */

char *help; /* Help message (long) */

/* do auto completion on the arguments */

int (*complete)(int argc, char * const argv[], char last_char, int maxv, char *cmdv[]);

};

typedef struct cmd_tbl_s cmd_tbl_t;

Note that some conditional compilation directives are omitted to improve readability.

2. argc – the usual meaning, number of arguments for the command.
3. argv – the usual meaning, array of pointers, each of which points to an argument string

Note that all the commands listed in the cmd/ directory have their implementation function which have the same signature. i.e., all of them take the same set of arguments discussed above and returns int indicating whether the command was executed successfully or not.

Just declaring a function won’t make it a part of the U-boot executable. The compiler can optimize out a function which is not referenced anywhere else in the source code. Further, we have made this function unavailable outside the translation unit, making it impossible to access it by its name. This is where the U_BOOT_CMD() macro helps. U_BOOT_CMD() macro generates a structure of type cmd_tbl_t, which holds the specifics of this command and allocates memory to this structure in a new section of the U-boot binary. More on this “linker list” mechanism later.

The echo command gets compiled by specifying an entry in the Makefile. The Makefile in cmd/ directory creates an object file from “echo.c”, if the CONFIG_CMD_ECHO is defined -

obj-$(CONFIG_CMD_ECHO) += echo.o

To summarize, here is the workflow to add a new command –

1. Create <my_command>.c file in cmd/

2. Write the do_<my_command> function having the same signature discussed above.

3. Use the U_BOOT_CMD() macro and register this command with the U-boot shell.

4. Edit the Makefile to generate an object file out of it.

obj-y += <my_command>.o

Example :

Let’s create a new command “echo2” which has the same functionality as “echo”, just to make the workflow clear. Here are the steps –

1. cd cmd

2. cp echo.c echo2.c

3. Edit echo2.c, change the name of command as given in U_BOOT_CMD macro.

4. Edit Makefile to add the following line

Obj-y += echo2.o

Now you need to test it. If you don’t have a hardware platform, you can use the sandbox feature to test your feature on your host machine. Alternatively, you can use any hardware platforms (eg: Beaglebone) or QEMU to emulate a hardware platform.

Lets build u-boot for the sandbox target. Go to the top-level u-boot directory and execute. (See boards/sandbox/README.sandbox for more details on this)

$make mrproper

$make sandbox_defconfig all NO_SDL=1

Now the binary can be executed as -

$./u-boot

U-Boot 2018.01-dirty (Mar 20 2018 - 18:10:22 +0530)

DRAM: 128 MiB

MMC:

Using default environment

In: serial

Out: serial

Err: serial

SCSI: Net: No ethernet found.

IDE: Bus 0: not available

Hit any key to stop autoboot: 0

Invalid host device number

=> echo2 Hello World!

Hello World!

Having verified that our new command is getting compiled into the u-boot binary, let’s understand the linker-list mechanism which binds all the commands into the u-boot binary, and the mechanism that the shell uses to invoke the command.

The need for Linker List

Think of how the U-boot shell functionality can be implemented as a C program. Assume that the functions to get the user input as a string, and separating out the command name and arguments are taken care of. A simple function which executes a command, given the command name may look like this (of course, we have left out passing arguments and getting back the return values) –

int execute_command(const char* cmd_name)

{

Int return_val;

if (strcmp(cmd_name, “echo”) == 0)

return_val = do_echo();

else if(strcmp(cmd_name, “bdinfo”) == 0)

return_val = do_bdindo();

else if(…)

…

Return return_val;

}

Though it looks simple, it is not maintainable. When you need to introduce a new command, a new_command.c file has to be created, and a new else if () statement has to be added in another file. Although we can use the conditional compilation features and macros to handle various architecture, board, and configuration-specific commands, we will end up declaring a lot of macros which will be difficult to keep track of.

Let’s make it better. Here we define a structure which binds the command name and the function which invokes it together.

struct command_entry_s

{

Char* name;

int (*do_cmd)(void);

};

typedef struct command_entry_s command_entry_t;

We can use the above structure and create an array of it. This enables traversing the array to find the command entered by the user, as shown.

command_t cmd_list[] =

{

{ .name = ”echo”, .do_cmd = do_echo},

{.name = “bdinfo”, .do_cmd = do_bdinfo},

…

}

int execute_command(const char* cmd_name)

{

Int return_val;

Command_t cmd_temp;

Int I;

Int cmd_list_length = sizeof(cmd_list) / sizeof(cmd_temp);

For(i=0; i<length; i++)

{

Cmd_temp = cmd_list[i];

If( strcmp(cmd_temp.name, cmd_name) == 0)

{

Return_val = Cmd_temp.do_cmd();

Break;

}

Return return_val;

}

This example is much more maintainable compared to the previous. When you need to add a new command, you need to add a new entry to the cmd_list array, which can be kept as a separate file. Further you can keep come array entries to be conditionally compiled(using #ifdef X .. #endif). Still, this involves adding a new file for the command, and editing part of the source code.

The concept of linker list is very similar to the above example, but is more maintainable.

The Linker List explained

The cmd_tbl_t structure and the function which implements the command functionality gives all that is needed to execute the command in run-time. Earlier we have seen that the U_BOOT_CMD() macro adds a new command to the U-boot shell. Let’s see how this macro gets expanded for ‘bdinfo’ command.

/cmd/bdinfo.c has –

U_BOOT_CMD(

bdinfo, 1, 1, do_bdinfo,

"print Board Info structure",

);

Replaced by –

U_BOOT_CMD_COMPLETE( bdinfo, 1, 1, do_bdinfo, "print Board Info structure", “”, NULL);

U_BOOT_CMD() is defined as –

#define U_BOOT_CMD(_name, _maxargs, _rep, _cmd, _usage, _help) \

U_BOOT_CMD_COMPLETE(_name, _maxargs, _rep, _cmd, _usage, _help, NULL)

Replaced by –

ll_entry_declare(cmd_tbl_t, bdinfo, cmd) = U_BOOT_CMD_MKENT_COMPLETE(bdinfo, 1, 1, do_bdinfo, "print Board Info structure", “”, NULL);

U_BOOT_CMD_COMPLETE is defined as –

#define U_BOOT_CMD_COMPLETE(_name, _maxargs, _rep, _cmd, _usage, _help, _comp) \

ll_entry_declare(cmd_tbl_t, _name, cmd) = \

U_BOOT_CMD_MKENT_COMPLETE(_name, _maxargs, _rep, _cmd, \

_usage, _help, _comp);

Replaced by –

ll_entry_declare(cmd_tbl_t, bdinfo, cmd) = {“bdinfo”, 1, 1, do_bdinfo, "print Board Info structure", _CMD_HELP(“”), _CMD_COMPLETE(NULL)};

U_BOOT_CMD_MKENT_COMPLETE() is defined as –

#define U_BOOT_CMD_MKENT_COMPLETE(_name, _maxargs, _rep, _cmd, \

_usage, _help, _comp) \

{ #_name, _maxargs, _rep, _cmd, _usage, \

_CMD_HELP(_help) _CMD_COMPLETE(_comp) }

Replaced by –

ll_entry_declare(cmd_tbl_t, bdinfo, cmd) = {“bdinfo”, 1, 1, do_bdinfo, "print Board Info structure", “”, NULL};

_CMD_HELP() is defined as –

#ifdef CONFIG_SYS_LONGHELP

# define _CMD_HELP(x) x,

#else

# define _CMD_HELP(x)

#endif

_CMD_COMPLETE() is defined as –

#ifdef CONFIG_AUTO_COMPLETE

# define _CMD_COMPLETE(x) x,

#else

# define _CMD_COMPLETE(x)

#endif

Replaced by –

cmd_tbl_t _u_boot_list_2_cmd_2_bdinfo __aligned(4) __attribute__((unused, section(“.u_boot_list_2_cmd_2_bdinfo”))) = {“bdinfo”, 1, 1, do_bdinfo, "print Board Info structure", “”, NULL};

ll_entry_delcare() is defined as –

#define ll_entry_declare(_type, _name, _list) \

_type _u_boot_list_2_##_list##_2_##_name __aligned(4) \

__attribute__((unused, \

section(".u_boot_list_2_"#_list"_2_"#_name)))

After pre-processing, the macro expansion causes a variable of type “cmd_tb_t” to be declared, with the variable name obtained by concatenating “_u_boot_list_2_”, “cmd”,“_2_” and command name. Note the use of token concatenating (##) and stringizing (#) operators to results in a unique symbol name. The compiler directive __attribute__((unused, section(“.section_name”))) makes sure that this variable will be included during linking, though it may not be used. It also creates a section in the program with a unique name similar to the name of variable.

For every command in cmd/ directory, the macro generates a variable of type cmd_tbl_t, with a unique name and unique section to keep that variable. For ‘N’ different variables, there are ‘N’ such sections. To view the section that linker has created, examine the “u-boot.map” file generated after the build.

The linker program, ld, takes all the object files and generate the final executable. The sections defined in the object files of the command are input sections to the linker. The linker takes all the input sections and generates corresponding sections in the final executable. Linker script controls the behavior of the linker. There is a linker script named “u-boot.lds” per board, located at board/vendor/board_name/ (e.g.: /board/ti/am335x). A part of this linker script reads –

. = ALIGN(4);

.u_boot_list : {

KEEP(*(SORT(.u_boot_list*)));

}

This means that the linker script creates a section “.u_boot_list” in the final executable, and places in it all the sections from input files (object files) which start “.u_boot_list” in an alphabetically sorted order. The sections of the the binary can be examined using objdump ($objdump -h u-boot). In addition, there are two special sections generated in the binary to mark the start and end of the list - ".u_boot_list_2_cmd_1" marks the start, and ".u_boot_list_2_cmd_3" marks the end. At the end, the sorted list of sections in the executable will be in the order –

".u_boot_list_2_cmd_1"

“.u_boot_list_2_cmd_2_bdinfo”

“.u_boot_list_2_cmd_2_echo”

…

".u_boot_list_2_cmd_3"

See the content of u-boot.map which describes the symbols defined under the “.u_boot_list” section

.u_boot_list 0x0000000000721148 0x8950

*(SORT(.u_boot_list*))

.u_boot_list_2_cmd_1

0x0000000000721148 0x0 cmd/built-in.o

.u_boot_list_2_cmd_1

0x0000000000721148 0x0 common/built-in.o

.u_boot_list_2_cmd_2_askenv

0x0000000000721148 0x30 cmd/built-in.o

0x0000000000721148 _u_boot_list_2_cmd_2_askenv

.u_boot_list_2_cmd_2_base

0x0000000000721178 0x30 cmd/built-in.o

0x0000000000721178 _u_boot_list_2_cmd_2_base

.u_boot_list_2_cmd_2_bdinfo

0x00000000007211a8 0x30 cmd/built-in.o

0x00000000007211a8 _u_boot_list_2_cmd_2_bdinfo

……

…….

.u_boot_list_2_cmd_2_ut_print

0x0000000000722bb8 0x30 test/built-in.o

0x0000000000722bb8 _u_boot_list_2_cmd_2_ut_print

.u_boot_list_2_cmd_2_version

0x0000000000722be8 0x30 cmd/built-in.o

0x0000000000722be8 _u_boot_list_2_cmd_2_version

.u_boot_list_2_cmd_3

0x0000000000722c18 0x0 cmd/built-in.o

How U-boot shell uses the Linker List to find commands?

When the user types in a command to the u-boot shell, string operations are performed on it to separate the command name, number of arguments(argc) and array of pointers to each of the arguments(argv). The command name is then passed to the function find_cmd(), which returns the pointer to the cmd_tbl_t structure which holds the details of the command.

cmd_tbl_t *find_cmd(const char *cmd)

{

cmd_tbl_t *start = ll_entry_start(cmd_tbl_t, cmd);

const int len = ll_entry_count(cmd_tbl_t, cmd);

return find_cmd_tbl(cmd, start, len);

}

Let’s understand how the table is traversed to find the command entered by the user.

The ll_entry_start(cmd_tbl_t, cmd) macro returns the address of the first variable in the list of u-boot commands, which is allocated in the section labelled ".u_boot_list_2_cmd_1" which we have declared earlier.

#define ll_entry_start(_type, _list)

({

static char start[0] __aligned(4) __attribute__((unused,

section(".u_boot_list_2_"#_list"_1")));

(_type *)&start;

})

Similarly, the variable allocated in the section labelled ".u_boot_list_2_cmd_3" marks the end of the list of commands.

#define ll_entry_end(_type, _list)

({

static char end[0] __aligned(4) __attribute__((unused,

section(".u_boot_list_2_"#_list"_3")));

(_type *)&end;

})

The difference between pointer values returned by ll_entry_start() and ll_entry_end() gives the total number of commands in the linker list.

#define ll_entry_count(_type, _list)

({

_type *start = ll_entry_start(_type, _list);

_type *end = ll_entry_end(_type, _list);

unsigned int _ll_result = end - start;

_ll_result;

})

Now we have the start address and length of the linker list containing the list of commands. This list can now be traversed to find the command we are looking for, by comparing the cmd_tbl_t->name string with the command string obtained from shell. This is shown below –

/* find command table entry for a command */

cmd_tbl_t *find_cmd_tbl(const char *cmd, cmd_tbl_t *table, int table_len)

{

#ifdef CONFIG_CMDLINE

cmd_tbl_t *cmdtp;

cmd_tbl_t *cmdtp_temp = table; /* Init value */

const char *p;

int len;

int n_found = 0;

if (!cmd)

return NULL;

* Some commands allow length modifiers (like "cp.b");

* compare command name only until first dot.

len = ((p = strchr(cmd, '.')) == NULL) ? strlen (cmd) : (p - cmd);

for (cmdtp = table; cmdtp != table + table_len; cmdtp++) {

if (strncmp(cmd, cmdtp->name, len) == 0) {

if (len == strlen(cmdtp->name))

return cmdtp; /* full match */

cmdtp_temp = cmdtp; /* abbreviated command ? */

n_found++;

}

if (n_found == 1) { /* exactly one match */

return cmdtp_temp;

}

#endif /* CONFIG_CMDLINE */

return NULL; /* not found or ambiguous command */

}

Conclusion

In this post, the working of linker list was explored, along with the use-case of how it can be used to bind shell commands with u-boot binary. It also demonstrated the maintainability of commands in u-boot just by adding a single “command.c” file and an entry in the corresponding Makefile. This understanding of Linker List will become handy while discussing the driver model in u-boot, which decouples drivers from the main u-boot source code, and enables dynamic probing.

References –

1. doc/README.commands, this readme gives the workflow to add a command, and briefly describes the linker list feature
2. include/linker_lists.h, describes all the macros used. The comments provide an insight into how linker_list mechanism works.
3.https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes , describes variable attributes available in gcc
4. An Introduction to the GNU Compiler and Linker, William Gatliff, gives a good idea about linker and linker scripts.

My U-Boot Experiments

Search This Blog