Jarvis's Blog

白帽子、全栈、IoT安全专家、嵌入式系统专家

在Android中使用ARM Coresight ETM/ETE进行指令流分析的方法及平台选型

关于CoreSight

这篇文章不打算介绍CoreSight基本原理,本文假设读者已经对CoreSight的基本原理以及作用有了基本了解,打算使用CoreSight来做TA的分析或者App及内核的高效Trace和重放。本文主要给出一种获得容易获得的Coresight ETM/ETE设备平台的渠道。

对于Coresight相关的基本介绍,原理,可以参考这些文章:

coresight(一)coresight简介

coresight(二)coresight寄存器

coresight(三)APB,ATB总线

coresight(四) channel interface

coresight(五) rom table

coresight(六) power requestor

coresight(七)coresight的两大功能

coresight(八)soc-400套件

CoreSight在ARM TA安全研究中的应用论文:

Google的文档关于如何在Android中开启Coresight ETM以及ETM在AutoFDO中的应用:

https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/collect_etm_data_for_autofdo.md

Linux基金会在内核中使用MEM-AP整合Coresight ETM所做的工作:

https://www.slideshare.net/linaroorg/lcu14-101-coresight-overview

https://elinux.org/images/b/b3/Hardware_Assisted_Tracing_on_ARM.pdf

支持CoreSight的硬件选型

虽然理论上来说,只要是ARM平台,Coresight ETM宏单元是公版ARM核心的一部分,只要是ARM架构都可以使用,但实际情况并非如此,以下我们分硬件类别来说明:

服务器平台

目前常见的ARM服务器平台有:

华为鲲鹏服务器 华为鲲鹏服务器(以鲲鹏920为代表)明确支持Coresight ETM,可以在Linux中对程序进行动态Trace,但可惜的是,目前华为并不开放Coresight驱动,以至于外部用户无法使用。

Marvell ThunderX2服务器 明确支持Coresight ETM,也有相关的驱动和开源案例,但由于该平台很小众,已停产,买不着。

Ampere Computing服务器产品 以Ampere Altra和AmpereOne架构为代表,支持Coresight ETM,但目前根据笔者测试,虽然Ampere官方放出了了Coresight相关的驱动,但问题在于,目前各大厂商主板的UEFI固件并未将Coresight硬件通过ACPI协议报告给上层系统(可能因为觉得这个功能没什么人使用),因此目前仍然无法使用。

开发板

RockChip RK3399开发板 根据Rockchip官方的介绍,RK3399芯片是支持完整的Coresight ETM功能的(RK3588/3588S等后续产品阉割了这部分支持),但目前并没有开源的设备树启用它,因此暂时没有现成方案,如果要使用需要自己适配。

NVIDIA Jetson TX2开发板 明确支持,可用。

NVIDIA Jetson Nano开发板 明确支持,可用。

NVIDIA平台已有开源拿项目可以拿来用,参考:https://github.com/RICSecLab/coresight-trace

手机

根据笔者的探索,高通方案的手机一直支持Coresight ETM功能(从8gen1使用armv9架构后,ETM被同功能的ETE单元取代),但很遗憾,量产机型都对SoC开启了Secureboot(启用secureboot的过程被称之为熔断),也就是熔断,熔断之后的机器,其Debug功能会被关闭,如果要开启,需要通过熔断时配置的证书签名的Debug Policy来启用。但量产机型中,证书掌握在OEM厂商手中,一般不会泄漏出来,因此ETM功能处于不可用的状态。

对于Coresight的认证过程根据ARM官方文档的资料,CoreSight DEBUG和Trace模块接受4个个引脚的控制:DBGEN,SPIDEN,NIDEN,SPNIDEN。

但这几个信号很多时候并不是以引脚的形式留在外面,查阅了一些资料,其信号使能方式和SoC的具体设计有关系,因为ARM仅仅提供了Core的实现,而芯片要工作,往往需要加上外围的IO模块,因此各家厂商配用的模块不同,最终的设计可能不同。但由于各厂商对认证的方式公开资料较少,在这里只整理了一些已知的信息。

  • 高通平台:当soc未配置secureboot时,debug权限默认开启,当secureboot开启后,debug就会被关闭,此时需要oem厂商签名的debug policy镜像文件来通过auth认证开启权限。
  • 三星Exynos平台:系列使用SJTAG模块,challenge response方式认证,本质还是证书。
  • 飞思卡尔iMX系列:这些信号接在DAP模块上,需要外部调试器操作DAP寄存器来启用这些信号。
  • Google Silicon GS系列:认证方案和三星Exynos系列类似,推测GS系列芯片是三星Exynos的半定制产品

下面着重比较各类可用的平台

Pixel 系列工程机

目前能较方便获得的机型主要就只有Pixel 4/4 XL的EVT/DVT机型其他机型极少流出

以下是笔者测试的一台MP量产版Pixel 3a XL,即使在内核中编译了Coresight ETM,驱动,可见其AUTH寄存器仍然是0x88,也即debug功能(包括Secure world和non-Secure World当中的Trace功能)属于关闭状态,当Debug功能可用时,该寄存器的值应该为0xcc。

参考/sys/bus/coresight/devices/coresight-etm0/mgmt/trcauthstatus寄存器的值。(以下是量产版MP机型Pixel 3a XL的结果)

因此,如果要在高通平台的手机上快乐的使用ETM,需要满足两个条件:

  • 非熔断机器:例如方案验证阶段的机型,比如EVT/DVT阶段的工程机(一旦进入PVT和MP阶段,就会熔断,此时已不可用),例如最方便的Pixel 4工程机(量大,目前仍可买到二手),高通原厂参考设计,例如QRD,MTP,HDK等机型。
  • 与机型配套的源代码(至少需要有内核源代码以添加Coresight ETM设备树和驱动),这样的条件,Pixel由于Google给出源码,因此可以满足条件,而高通的QRD/MTP/HDK机型,如果有高通授权的QTI账号,或者特殊渠道,也可以获得相关源码。

对于Pixel 4的非熔断机器(EVT DVT),可以清楚的看见Secureboot为NONE,即没有开启

从系统中也可以清楚的看到,trcauthstatus的值为0xcc,即ETM为开启状态。

但可惜的是,目前Pixel系列的工程机,仅能买到n手Pixel 4或者Pixel 4 XL,且内核版本在4.9~4.14不等,而如果要结合ebpf分析,无法使用5.10以后版本内核中的新patch(安卓系统使用的高通内核无法升级到更高版本)。

高通QRD/MTP/HDK工程机

高通原厂工程机,可以获得最新的SoC,且没有熔断,而且内核源码一般都会开放(例如小米,oneplus或者OPPO一般都会在github上开源内核源码),可自行定制功能。有多种机型、平台可以选择,数量也可保证,如果你对高通工程机感兴趣,欢迎交流。

QRD888:

QRD8450(8 gen 1)

MTP8550(8 gen 2)

QRD/MTP/HDK机型均未熔断,Secureboot为关闭状态,此时ETM可用:

MTP8450(8 gen 1)

/sys/bus/coresight/devices/coresight-ete0/mgmt/trcauthstatus的值为0xcc可用

同时可以使用simpleperf进行trace或者inject:

taro:/data/local/tmp # simpleperf record -e cs-etm --duration 3 -a
simpleperf W dso.cpp:434] failed to read symbols from [qcom_hwspinlock]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [smem]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_wdt_core]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_virt_wdt]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_ipc_logging]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [cmd_db]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_rpmh]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [proxy_consumer]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gdsc_regulator]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [icc_bcm_voter]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_ipcc]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_tsens]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [rpmh_regulator]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [dcvs_fp]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [icc_rpmh]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_dcvs]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_cpufreq_hw]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [thermal_pause]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [cfg80211]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [hwkm]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [crypto_qti_hwkm]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [dispcc_waipio]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gcc_diwali]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_arm_drv]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [iommu_logger]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [llcc_qcom]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [secure_buffer]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [mem_buf]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qti_fixed_regulator]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [reboot_mode]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [msm_geni_se]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gcc_waipio]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [phy_qcom_ufs]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pinctrl_diwali]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_dload_mode]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pmu_vendor]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [regmap_spmi]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_spmi_pmic]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qnoc_diwali]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [socinfo]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [stub_regulator]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [ufshcd_crypto_qti]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [mac80211]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_rimps]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [ufs_qcom]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [minidump]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_dbl]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [phy_qcom_ufs_qmp_v4_cape]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qnoc_qos]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_reboot_reason]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_llcc_pmu]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [clk_qcom]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_rm_drv]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [c1dcvs_scmi]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_cpu_vendor_hooks]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [sched_walt_debug]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qrtr]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [mem_offline]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_scm]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [phy_generic]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [phy_qcom_ufs_qmp_v4_diwali]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [msm_qmp]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_msgq]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [mem_buf_dev]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_pdc]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_dma_heaps]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [clk_rpmh]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [clk_dummy]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qti_regmap_debugfs]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [icc_debug]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_iommu_util]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [phy_qcom_ufs_qmp_v4_waipio]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_pmu_lib]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [msm_rtb]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [spmi_pmic_arb]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [rtc_pm8xxx]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [nfc_i2c]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_aoss]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qnoc_waipio]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qcom_gic_intr_routing]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [gh_ctrl]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [debug_regulator]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [cqhci]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [dispcc_diwali]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pinctrl_msm]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [bwmon]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pinctrl_cape]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [mem_hooks]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [sched_walt]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [nvmem_qcom_spmi_sdam]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [tmecom_intf]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [crypto_qti_common]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [arm_smmu]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pmu_scmi]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [msm_dma_iommu_mapping]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [qca6490]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [msm_geni_serial]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [memory_dump_v2]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [cpu_hotplug]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [pinctrl_waipio]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [kryo_arm64_edac]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [c1dcvs_vendor]: File not found
simpleperf W dso.cpp:434] failed to read symbols from [bcl_pmic5]: File not found
simpleperf I cmd_record.cpp:750] Aux data traced: 58612288
taro:/data/local/tmp #

曲面屏贴膜选择指引

上一篇

Windows Dev Kit 2023(WDK2023) 安装Linux (Ubuntu 23.04)操作系统(附镜像)

下一篇
评论
发表评论 说点什么
  • 大佬,想问下,有没有什么办法知道ARM SOC是否支持Coresight ETM?可以通过某些寄存器来查询吗?

    • 一般都是支持的,取决于官方文档有没有告诉你寄存器的位置。以及有没有提供MEM-AP的驱动,一般单片机只提供了DAP

1115
0

    浙公网安备 33011002014706号