devspec0ps

AWS PTP Hardware clock

devspec0ps's photo
devspec0ps
·

2 min read

# IN CASE OF Skipping BTF generation for /home/dm-admin/amzn-drivers/kernel/linux/ena/ena.ko due to unavailability
of vmlinux when making

sudo apt install dwarves

sudo cp /sys/kernel/btf/vmlinux /usr/lib/modules/`uname -r`/build/

Supported only in Tokyo region on r7g.* instances.

Log in with EC2 Serial console, you need to remove the current ena kernel module and that will disrupt all connections.

git clone https://github.com/amzn/amzn-drivers.git

cd amzn-drivers/kernel/linux/ena

Compile ena module with PHC

ENA_PHC_INCLUDE=1 make

Install and Load the Custom Module: Now that you have successfully built the custom ENA module, you need to install and load it:

  • Backup the existing ena.ko module: sudo cp /lib/modules/$(uname -r)/kernel/drivers/net/ethernet/amazon/ena/ena.ko ~/ena.ko.backup

  • Copy your new module: sudo cp ena.ko /lib/modules/$(uname -r)/kernel/drivers/net/ethernet/amazon/ena/

  • Update module dependencies: sudo depmod -a

  • Load the module: sudo modprobe -r ena && sudo modprobe ena phc_enable=1

Check that the ena module was replace with the PHC enabled one (check last line):

dm-admin@dm-arm-trader:~/amzn-drivers/kernel/linux/ena$ modinfo ena
filename:       /lib/modules/5.15.0-1051-aws/kernel/drivers/net/ethernet/amazon/ena/ena.ko
version:        2.11.0g
license:        GPL
description:    Elastic Network Adapter (ENA)
author:         Amazon.com, Inc. or its affiliates
srcversion:     4A65CC132E6C115C57322E9
alias:          pci:v00001D0Fd0000EC21sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd0000EC20sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00001EC2sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00000EC2sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00000051sv*sd*bc*sc*i*
depends:        
name:           ena
vermagic:       5.15.0-1051-aws SMP mod_unload modversions aarch64
parm:           debug:Debug level (-1=default,0=none,...,16=all) (int)
parm:           rx_queue_size:Rx queue size. The size should be a power of 2. Depending on instance type, max value can be up to 16K
 (int)
parm:           force_large_llq_header:Increases maximum supported header size in LLQ mode to 224 bytes, while reducing the maximum TX queue size by half.
 (int)
parm:           num_io_queues:Sets number of RX/TX queues to allocate to device. The maximum value depends on the device and number of online CPUs.
 (int)
parm:           enable_bql:Enable BQL.
 (int)
parm:           lpc_size:Each local page cache (lpc) holds N * 1024 pages. This parameter sets N which is rounded up to a multiplier of 2. If zero, the page cache is disabled. Max: 32
 (uint)
parm:           phc_enable:Enable PHC.
 (uint)

Now you MUST see something in:

ls /sys/class/ptp/

for persistence:

sudo vi /etc/modules-load.d/ena.conf

Inside:

ena

sudo vi /etc/modprobe.d/ena.conf

Inside:

options ena phc_enable=1

sudo update-grub
sudo update-grub2

If no persistence again: unload module and load it again and then sudo depmod -a

Update chrony sources (make sure the chrony.conf is at that path)

sudo sh -c 'echo "refclock PHC /dev/ptp0 poll 0 delay 0.000010 prefer" >> /etc/chrony/chrony.conf'

sudo systemctl restart chronyd sudo chronyc sources

AWS