Designing bootloader for Microchip dsPIC33E/PIC24E micro-controller (2)

In my last post “Designing bootloader for Microchip dsPIC33E/PIC24E micro-controller (1)“, I talked about some basics of bootloader and picked up one of the architectures as my solution, which is to place the bootloader towards the end of the on-board flash. In this post, I will go over the work flow of bootloading, hoping to give you an image of how the whole things work out.

After you finish your embedded application design, the compiler will compile your C/C++/Assembly codes into binary, which is usually output under the extension name “hex” in a format of INTEL HEX32. I will talk about the Intel Hex32 format in details later on. Then you open your hex loader software on PC and load in the hex file. Before starting burning the binary into your micro-controller, make sure your programmer (ICD2 or ICD3 for Microchip) is properly connected to your target device, and the compiled codes will be downloaded into the device through PGC/PGD ports after you click the execution button. Microchip has built all the components in the tool chain: compiler (MPLAB IDE X) – hex loader (MPLAB IPE X) – in circuit programmer (ICD2/ICD3) – hardware receiver on the device (PGC/PGD ports). In order to make the bootloader work, you will have to build some of the components in the tool chain for your own, which will probably look like: compiler (MPLAB IDE X) – your own hex loader – your own software/hardware programmer responsible for transmitting the binary through peripherals supported by the device such as USB, UART, SPI, I2C or even SD card – software receiver, which is your bootloader codes that implemented with some peripherals. Yes, as you probably already figured out, designing your own bootloader is not jut the piece of codes hiding in your micro-controller.

3. Understand the standard INTEL HEX32 format
If you open the compiled “.hex” file in hex format, you will find nothing but lines after lines of hex string. All of the lines follow this single format:
:BBAAAATTHHHH….HHHHCC, where:

[:] Beginning mark, always as “:”
[BB] Number of bytes in HHH…HH part
[AAAA] Starting address
[TT] Type:
- 00: Data Record
- 01: End Record
- 02: Extended Segment Address Record
- 03: Start Segment Address Record
- 04: Extended Linear Address Record
- 05: Start Linear Address Record
[HHHH….HHHH] Data
[CC] Checksum

I won’t cover the details of this format here as they are everywhere on the Internet. For example: Microchip DS70619B: dsPIC33E/PIC24E Flash Programming Specification. Instead, I will spend some efforts here looking into how it is implemented on the dsPIC33E/PCI24E device, and putting together all the related details that are scattered across numerous documents. Let’s start with some example hex lines to strengthen your understanding:

:020000040108EA
:0200000212FFBD
:0401000090FFAA5502
:00000001FF

Determine the extended linear address offset for the data record (0108 in this example).
```
:020000040108EA
```
Determine the extended segment address for the data record (12FF in this example).
```
:0200000212FFBD
```
Determine the address offset for the data in the data record (0100 in this example).
```
:0401000090FFAA5502
```
Calculate the absolute address for the first byte of the data record.

+ 0108 0000 linear address offset
shifted left 16 bits

+ 0001 2FF0 segment address offset
shifted left 4 bits

+ 0000 0100 address offset from
data record

= 0109 30F0 32 bit address for
first data byte

Which gives us the following

010930F0 90
010930F1 FF
010930F2 AA
010930F3 55

Please note that as soon as data type “02” or “04” appears, all the following lines’ address will be shifted until another “02” or “04” line shows up, then a new shift should be re-caculated.

4. Understand dsPIC33E/PIC24E specific INTEL HEX32 format
Microchip added some of its own rules on top of standard INTEL HEX32 format. They must be understood before a microchip hex file could be correctly translated. That’s what really confused me during the early stage of the bootloader development.

Each instruction for dsPIC33E/PIC24E is 24-bit or 3-byte wide, and flash program memory is also 24-bit wide. However, in the standard INTEL HEX32, data always come as multiple of 4 bytes. In order to accommodate INTEL’s rule. Microchip throw away the right most byte of every 4 bytes in the hex file. This byte is called “phantom byte”. That’s why you can find in the hex file that the right most byte of every 4 bytes is always “00”. So divide the “BB” part by 4 is the count of instructions contained on that line.
Address in the flash memory is incremented by 2 to read/write one instruction. This is because one 3-byte instruction takes two address if the address is based on 16-bit/2-byte width. Microchip explains this in their DS70609D as “The 24-bit Flash program memory can be regarded as two side-by-side 16-bit spaces, with each space sharing the same address range … the upper byte of Flash program memory that does not exist. This byte is called the ‘phantom byte.” This also explains why dsPIC33EP512MU810 has “User Memory Address Limit” up to 0x557FE (dec. 350,206) can only support 175,104 (350,208 / 2) instructions.
“AAAA” part (MSB-LSB) of Hex32 represents the starting address of the data. Divide this number by two is the real device address.
To understand GOTO and RESET in the Hex23 file:
The very second line of a hex file is always the line contains the GOTO and RESET instruction. It tells the chip where to find the first line of code after the chip is powered or reset. It contains only two instructions with address at 0. The first instruction, usually ends with “0x04” (not counting the phantom byte), means “Jump to address low 16-bit”. The first 2 bytes are the address. the second instruction, usually ends with “0x00”, means “Jump to address high 16-bit”. The first 2 bytes are the address.
Here is an example：
```
:0800000000a80400020000004a
:08 0000 00 00a80400 02000000 4a
 (1)(2)  (3)(4)      (5)      (6)
```
(1). There are 8 bytes on this line
(2). Starting address 0
(3). Type: data record
(4). 1st instruction: 00 a8 04 00(phantom byte) -> (04)jump lower 16-bit address(a800)
(5). 2nd instruction: 02 00 00 00(phantom byte) -> (00)jump higher 16-bit address(0020)
(6). CRC
This line represents: Jump to address 0x02a800.

Another example:
```
:080000000002040000000000f2
```
Represents: Jump to address 0x200. (This hex is normal as it jumps to the beginning of user program flash memory of e.g. dsPIC33FJ256GP710A)