Intel-PT does not record any packets when KVM-QEMU is on

2
votes

I am trying to use Intel-PT on the host, while I run a general software program in the guest machine. So what I expect is the Intel-PT running in the host will record all the relevant packets(like PIP, FUP, TSC etc.) and also all the VM-based packets like VMCS.

I use the below command -

./perf kvm --host --guest --guestkallsyms=guest-kallsyms --guestmodules=guest-modules record -e intel_pt//

guest-kallsyms and guest-modules are the kallsyms and module files I copied from the guest onto my host.

I will then start my Virtual Machine. I will run a program on the guest machine. Once the program execution is complete, I will press Ctrl + C (SIGINT) in my host to stop the recording.

I see that once I try to use perf report to read the file generated using the below command -

./perf kvm report -i perf.data.kvm

It returns "NO SAMPLES FOUND". This means that Intel-PT has failed to record any samples.

NOTE: I found that the bit 14 for the value in the MSR MSR_IA32_VMX_MISC is 0, for my processor. As per the Intel documentation, this bit should be 1 for Intel-PT to be used in VMX operation. Does this in any way affect why Intel-PT does not record any samples ?

Will INTEL-PT work even when the VM is on ? Or is my method of recording data wrong ?

Edit: I am using Linux Kernel 4.11.3, having Ubuntu 17.04 and a Broadwell CPU, which supports Intel-PT.

1
linuxvirtualizationkvmtraceperf
I think Intel PT has to be enabled in the UEFI due to its propensity to cause security issues. Have you confirmed the processor supports it and it is enabled? Also, you might need to run your command as root if PT is available and enabled. (I thought I read an Intel whitepaper on it, but I can't find it at the moment). – jww
Hi @jww, thanks for the information. I have PT available and enabled as I have profiled normal userspace programs previously with it. However, I ran the command as "sudo" as you said, I have interestingly run into a segmentation fault. – Arnabjyoti Kalita
PT can't be used with virtualization enabled. I can't say right now if it's a limitation for all CPUs, or only for early implementation like Broadwell. – 0andriy
Yes @0andriy, that is what I thought initially. But I have read about people performing PT with KVM, but that was on a newer Skylake processor. I do not know if the 14th bit of the MSR_IA32_VMX_MISC being reset is the problem actually. – Arnabjyoti Kalita
patchwork.kernel.org/patch/8726591 – 0andriy

1 Answers

5
votes

Since I now have a clear idea of why Intel-PT does not work with QEMU-KVM on, I will post an answer.

As I mentioned in the question, the main reason for this not to work is the fact that the bit 14 for the value in the MSR MSR_IA32_VMX_MISC is 0, for my processor. As per the Intel documentation, this bit should be 1 for Intel-PT to be used in VMX root operation(between VMXON and VMXOFF).

The main problem is that when the above bit is 0, a VMXON instruction will set the TraceEn component of IA32_RTIT_CTL MSR to 0. This component controls the tracing operation, if this is reset, no tracing data is written to the buffer. This reset is controlled at the hardware level.

To perform this activity, it is necessary to have a Skylake processor, at least. I was using a Broadwell system, which, as it looks now, will not work.