What time is it? RTCs explained in embedded Linux

Posted by Marcus Folkesson on Tuesday, July 2, 2024

What time is it? RTCs explained in embedded Linux

Most smart devices keep track of time and it's usually something that everyone expects to be "right", accurate and just work. I mean, you can keep time pretty well with just a pendulum or a spring [1], so why would this be a complicated thing in an embedded system?

Your smart alarm clock, smart phone or even your new cool IoT refrigerator make all use of the time for sure, but they probably do it for different purposes and have different requirements of the accuracy of time.

How the system make use of the time could differ in several ways. Example on use cases could be:

  • Show the current time to an user
  • Make sure that a SSL certificates has not expired
  • Execute a task once per day

All these use cases has different requirements on:

  • How accurate the time is
  • How much it is allowed to drift
  • If it has to be synchronized to any external clock

For example, a device that show time better be accurate and does not allow that much drifting. A device that verify SSL certificates has to be synchronized with the "world time" to work. A device that just performs one task per day does not have to be synchronized with the rest of the world and the drift may not be a problem if the tolerances is high enough.

/media/linux-clocks.jpg

Clocks in a Linux system

Lets simplify things for now and say that there are just two types of clocks in a Linux system: Hardware and System clocks.

System clock

The system clock is part of the Linux kernel and the system time is incremented by a timer interrupt. The system time is basically the number of seconds since 00:00:00 Januari 1, 1970 UTC.

It is the system time that is used by for all time operations and timekeeping features. Whenever you run timedatectl or date, this is the time you get.

For example, see

1$ timedatectl
2               Local time: mån 2024-06-24 18:05:01 CEST
3           Universal time: mån 2024-06-24 16:05:01 UTC
4                 RTC time: mån 2024-06-24 16:05:26
5                Time zone: Europe/Stockholm (CEST, +0200)
6System clock synchronized: no
7              NTP service: inactive
8          RTC in local TZ: no
and
1$ date
2mån 24 jun 2024 18:05:05 CEST

The system clock does not have its own power source, in other words, if the system is powered off it will stop counting and reset. This means that the system time is reverted back to 1970 once the device is powered on again.

To know what time it is even when the system is powered off, we need an external time source to synchronize against. Such external time source could be a hardware clock.

Hardware clock

A hardware clock is an independent hardware device that has its own power supply, typically a battery or capacitor, that runs even when the device is unplugged or powered off. A more common name for the hardware clock is RTC, Real Time Clock. The device is usually either an internal module in the SoC or connected to an external bus like I2C, SMBUS or SPI.

hwclock is a handy tool to read out and manipulate the current time stored in the RTC. It also has the capability to calculate and adjust for drifting as we will see later on.

1$ hwclock
22024-06-24 18:06:57.664097+02:00

Even the timedatectl gives you the current RTC time:

1$ timedatectl
2               Local time: mån 2024-06-24 18:05:01 CEST
3           Universal time: mån 2024-06-24 16:05:01 UTC
4                 RTC time: mån 2024-06-24 16:05:26
5                Time zone: Europe/Stockholm (CEST, +0200)
6System clock synchronized: no
7              NTP service: inactive
8          RTC in local TZ: no

What about "RTC in local TZ: no"?

The time in the RTC could either be stored in UTC or local time.

I would recommend to always use UTC time unless you are looking for trouble [3] with time zones and daylight saving switches.

Also, the NTP synchronization does not work of the RTC operates in local time.

Keep the clocks synchronized

It is essential that the system clock is correct, and thus also the hardware clock. But what is a correct time? The term correct is relative like many other things. Without any external interaction, the clock will always be correct - the time is always now.

But as soon as we have to interact with any externals, e.g. users or other system, it is usually necessary to have a common perception if what time it is. The most common way to get an accurate time is asking a Network Time Protocol, NTP, (or PTP) server and then adjust the system time.

But how and when are the system time and RTC time synchronized?

Of course, you can do it manually, usually with hwclock, and sometimes that is the only option. But usually the kernel will synchronize the clock for you.

There are few kernel configurations that will help you with that:

CONFIG_RTC_HCTOSYS

config RTC_HCTOSYS
        bool "Set system time from RTC on startup and resume"
        default y
        help
          If you say yes here, the system time (wall clock) will be set using
          the value read from a specified RTC device. This is useful to avoid
          unnecessary fsck runs at boot time, and to network better.

CONFIG_RTC_HCTOSYS read the time from the RTC set by CONFIG_HCTOSYS_DEVICE during boot/resume and update the system time. You have probably seen a message in your kernel log:

1[    3.157560] rtc-pcf8523 1-0068: setting system clock to 2024-06-20T12:01:06 UTC (1718884866)

This is when the system time is updated with the RTC time from the rtc_hctosys initcall [9].

CONFIG_RTC_SYSTOHC

config RTC_SYSTOHC
        bool "Set the RTC time based on NTP synchronization"
        default y
        help
          If you say yes here, the system time (wall clock) will be stored
          in the RTC specified by RTC_HCTOSYS_DEVICE approximately every 11
          minutes if userspace reports synchronized NTP status.

No userspace software normally touches the RTC, instead, the software set the STA_UNSYNC [4] bit to clock_adjtime(2) and let the kernel update the RTC time approximaetly every 11 minutes [5].

RTC_HCTOSYS_DEVICE and RTC_SYSTOHC_DEVICE

These are the devices used by CONFIG_RTC_HCTOSYS and CONFIG_RTC_SYSTOHC which both defaults to rtc0:

config RTC_HCTOSYS_DEVICE
        string "RTC used to set the system time"
        depends on RTC_HCTOSYS
        default "rtc0"
        help
          The RTC device that will be used to (re)initialize the system
          clock, usually rtc0. Initialization is done when the system
          starts up, and when it resumes from a low power state. This
          device should record time in UTC, since the kernel won't do
          timezone correction.

          This clock should be battery-backed, so that it reads the correct
          time when the system boots from a power-off state. Otherwise, your
          system will need an external clock source (like an NTP server).

          If the clock you specify here is not battery backed, it may still
          be useful to reinitialize system time when resuming from system
          sleep states. Do not specify an RTC here unless it stays powered
          during all this system's supported sleep states.
config RTC_SYSTOHC_DEVICE
        string "RTC used to synchronize NTP adjustment"
        depends on RTC_SYSTOHC
        default RTC_HCTOSYS_DEVICE if RTC_HCTOSYS
        default "rtc0"
        help
          The RTC device used for NTP synchronization. The main difference
          between RTC_HCTOSYS_DEVICE and RTC_SYSTOHC_DEVICE is that this
          one can sleep when setting time, because it runs in the workqueue
          context.

But... what if the RTC is wrong?

That the system read the hardware clock upon boot and updates the system time is a nice feature.

But what if the RTC time is wrong? How would you know? Assume you are using systemd, the timesyncd service will perform a few sanity checks to determine if the system time is valid by compare against a few known timestamps.

The first attempt is to compare against the timestamp environment variable SOURCE_DATE_EPOCH [10]. The current time can't be before the time the code was compiled, right?

Another timestamp that timesyncd compare against is the modification date of /var/lib/systemd/timesync/clock. This file is touched whenever the system gets a new time from an external time server. The current time could not be before that time either.

So there are a few attempts to validate the current system time. If the current time appear to be before the latest timestamp, the timesyncd service will set the system time to whatever is the latest known time was.

What about time zones?

Actually, the kernel keeps track of the system timezone [2], but almost nobody cares about it. Most applications rather use the TZ variable or the /etc/localtime file to determine which timezone it is.

Keeping time without external synchronization

It is not always possible to have a NTP server within reach to ask what time it is every now and then. Sometimes we are on our own and must try to be as accurate as possible without any external input.

Lets face it, most hardware clocks is not that accurate. Usually the RTC chip is driven by a 32kHz crystal oscillator that may drift up to 20ppm. This will end up in a drift of several seconds each day. Sure, a better oscillator will be more accurate and you may even go for a TCXO (Temperature Compensated Crystal Oscillator) to get the temperature factor out of calculation.

It is actually possible to compensate for the drifting and get the time quite accurate even with oscillators that drifts a lot. The thing is that much of this inaccurancy is completely predictable - it gains or loses the same of time every day. Even if the crystal oscillator does have a inaccurency, the frequency is highly determined by its load capacitance - including both the external capacitors you may see close to the crystal, but also the capacitance created between the traces and planes in the PCB.

This is rather constant and so is therfor the drift. This systematic drift may be corrected.

The adjtime file

hwclock has the /etc/adjtime file for keeping historical information about the last time the clock was set and calibrated.

For example:

1hwclock --set

Will set the hardware clock to the current system time. At the same time hwclock will create and/or update the adjtime file with the current timestamps.

The adjtime file will look like this:

0.000000 1720036215 0.000000
1720036215
UTC

The file format is as follows:

<The systematic drift rate in seconds per day> <Number of seconds since 1969 UTC> <Zero (for compatibility reasons)>
<Last calibration time>
<"UTC" or "LOCAL">

After some time, lets say 2 days, you set the hardware clock again and update the drift calculation:

1hwclock --set --update drift

If the time differ +6 seconds, the systematic drift is 3 seconds per day. Everytime you run hwclock --adjust or systemd-timedated runs, it will take this systematic drift into account.

What about all other clocks?

One may say that a recent glibc and Linux kernel has a lot clocks, not just a system clock and a hardware clock.

Well, that is true, just look at the manpage for clock_gettime(3) [6] and you will see a list of them:

  • CLOCK_REALTIME

    A settable system-wide clock that measures real (i.e., wall-clock) time. Setting this clock requires appropriate privileges. This clock is affected by discontinuous jumps in the system time (e.g., if the system administrator manually changes the clock), and by frequency adjustments performed by NTP and similar applications via adjtime(3), adjtimex(2), clock_adjtime(2), and ntp_adjtime(3). This clock nor‐ mally counts the number of seconds since 1970-01-01 00:00:00 Coordinated Universal Time (UTC) except that it ignores leap seconds; near a leap second it is typically adjusted by NTP to stay roughly in sync with UTC.

  • CLOCK_REALTIME_ALARM (since Linux 3.0; Linux-specific)

    Like CLOCK_REALTIME, but not settable. See timer_create(2) for further details.

  • CLOCK_REALTIME_COARSE (since Linux 2.6.32; Linux-specific)

    A faster but less precise version of CLOCK_REALTIME. This clock is not settable. Use when you need very fast, but not fine-grained timestamps. Requires per-architecture support, and probably also archi‐ tecture support for this flag in the vdso(7).

  • CLOCK_TAI (since Linux 3.10; Linux-specific)

    A nonsettable system-wide clock derived from wall-clock time but counting leap seconds. This clock does not experience discontinuities or frequency adjustments caused by inserting leap seconds as CLOCK_REALTIME does.

    The acronym TAI refers to International Atomic Time.

  • CLOCK_MONOTONIC

    A nonsettable system-wide clock that represents monotonic time since—as described by POSIX—"some unspecified point in the past". On Linux, that point corresponds to the number of seconds that the system has been running since it was booted.

    The CLOCK_MONOTONIC clock is not affected by discontinuous jumps in the system time (e.g., if the system administrator manually changes the clock), but is affected by frequency adjustments. This clock does not count time that the system is suspended. All CLOCK_MONOTONIC variants guarantee that the time returned by consecutive calls will not go backwards, but successive calls may—depending on the archi‐ tecture—return identical (not-increased) time values.

  • CLOCK_MONOTONIC_COARSE (since Linux 2.6.32; Linux-specific)

    A faster but less precise version of CLOCK_MONOTONIC. Use when you need very fast, but not fine-grained timestamps. Requires per-architecture support, and probably also architecture support for this flag in the vdso(7).

  • CLOCK_MONOTONIC_RAW (since Linux 2.6.28; Linux-specific)

    Similar to CLOCK_MONOTONIC, but provides access to a raw hardware-based time that is not subject to frequency adjustments. This clock does not count time that the system is suspended.

  • CLOCK_BOOTTIME (since Linux 2.6.39; Linux-specific)

    A nonsettable system-wide clock that is identical to CLOCK_MONOTONIC, except that it also includes any time that the system is suspended. This allows applications to get a suspend-aware monotonic clock without having to deal with the complications of CLOCK_REALTIME, which may have discontinuities if the time is changed using settimeofday(2) or similar.

  • CLOCK_BOOTTIME_ALARM (since Linux 3.0; Linux-specific)

    Like CLOCK_BOOTTIME. See timer_create(2) for further details.

  • CLOCK_PROCESS_CPUTIME_ID (since Linux 2.6.12)

    This is a clock that measures CPU time consumed by this process (i.e., CPU time consumed by all threads in the process). On Linux, this clock is not settable.

  • CLOCK_THREAD_CPUTIME_ID (since Linux 2.6.12)

    This is a clock that measures CPU time consumed by this thread. On Linux, this clock is not settable.

These are POSIX clocks and is indeed clocks in a Linux system. These are specified in posix-timers.h [7].

CLOCK_MONOTONIC represents a monotonic clock that starts to count during the early system boot. All other clocks is just represented as an offset appended to this clock. Consider the following implementation for a few of them:

 1/**
 2 * ktime_get_real - get the real (wall-) time in ktime_t format
 3 */
 4static inline ktime_t ktime_get_real(void)
 5{
 6	return ktime_get_with_offset(TK_OFFS_REAL);
 7}
 8
 9/**
10 * ktime_get_boottime - Returns monotonic time since boot in ktime_t format
11 *
12 * This is similar to CLOCK_MONTONIC/ktime_get, but also includes the
13 * time spent in suspend.
14 */
15static inline ktime_t ktime_get_boottime(void)
16{
17	return ktime_get_with_offset(TK_OFFS_BOOT);
18}
19
20/**
21 * ktime_get_clocktai - Returns the TAI time of day in ktime_t format
22 */
23static inline ktime_t ktime_get_clocktai(void)
24{
25	return ktime_get_with_offset(TK_OFFS_TAI);
26}

As you can see, they all use ktime_get_with_offset() [8]:

 1ktime_t ktime_get_with_offset(enum tk_offsets offs)
 2{
 3	struct timekeeper *tk = &tk_core.timekeeper;
 4	unsigned int seq;
 5	ktime_t base, *offset = offsets[offs];
 6	u64 nsecs;
 7
 8	WARN_ON(timekeeping_suspended);
 9
10	do {
11		seq = read_seqcount_begin(&tk_core.seq);
12		base = ktime_add(tk->tkr_mono.base, *offset);
13		nsecs = timekeeping_get_ns(&tk->tkr_mono);
14
15	} while (read_seqcount_retry(&tk_core.seq, seq));
16
17	return ktime_add_ns(base, nsecs);
18
19}
20EXPORT_SYMBOL_GPL(ktime_get_with_offset);

Which read out the monotonic clock and add an offset.

So, adjusting time in Linux is basically changing a offset relative to the monotonic clock.