-
Notifications
You must be signed in to change notification settings - Fork 546
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new record filter tool for public release of traces #6662
Comments
Simple record_filter_t::record_filter_func_t filter that modifies a field of every trace_entry_t record in a trace and leverages record_filter to output such modified records onto a "filtered" output trace. Issue: #6662
Containing-register IDs can be >=256, hence their value does not fit in the allotted 8 bits per register operand of regdeps encoding. This was causing a memory corruption in instr_convert_to_isa_regdeps() where src_reg_used and dst_reg_used have only 256 elements and are laid out next to each other in memory. Writing to index >=256 into one was overwriting the other. Fix: remap containing-register IDs to virtual-register IDs starting from 0 for all architectures. We still have only up to 198 unique containing registers (max number of containing registers for AARCH64), so remapping them allows to fit them in 8 bits. In the re-mapping (from DR_REG_ to DR_REG_V) we exclude DR_REG_INVALID to avoid issues with opnd_t operations for registers. We introduce 2 new public APIs: dr_reg_to_virtual() and get_virtual_register_name(). We use dr_reg_to_virtual() in instr_convert_to_isa_regdeps() to avoid the issue mentioned above. We also re-introduce setting the size for register operands in instr_convert_to_isa_reg_deps() and decode_isa_regdeps() as instr_t.operation_size because DR_REG_V don't have predefined size. We added tests to check that DR_REG_ with IDs >=256 don't cause problems. Issue: #6662
Containing-register IDs can be >=256, hence their value does not fit in the allotted 8 bits per register operand of regdeps encoding. This was causing a memory corruption in instr_convert_to_isa_regdeps() where src_reg_used and dst_reg_used have only 256 elements and are laid out next to each other in memory. Writing to index >=256 into one was overwriting the other. Fix: remap containing-register IDs to virtual-register IDs starting from 0 for all architectures. We still have only up to 198 unique containing registers (max number of containing registers for AARCH64), so remapping allows to fit them in 8 bits. In the re-mapping (from DR_REG_ to DR_REG_V) we exclude DR_REG_INVALID and DR_REG_NULL to avoid issues with opnd_t operations for registers. We introduce a private routine dr_reg_to_virtual() to do the mapping from real ISA to virtual register. We use it in instr_convert_to_isa_regdeps() to avoid the issue mentioned above. We modified the get_register_name() public API to use the global dcontext and its ISA mode to determine whether to return a real register name or a virtual one. The signature of the API remained the same, but we document the use of the global dcontext in doxygen. We also re-introduce setting the size for register operands in instr_convert_to_isa_reg_deps() and decode_isa_regdeps() as instr_t.operation_size because not all DR_REG_V have a predefined size based on their enum value (e.g., reserved DR_REG_XMM enum values). We added tests to check that DR_REG_ with IDs >=256 don't cause problems. Issue: #6662
We'd like to raise a point of discussion about the use of The DynamoRIO user documentation uses R for reasons of convention and generality, e.g. A new or relatively inexperienced user may probably and intuitively but wrongly try something like this first:
Rather than the correct:
or e.g.
This possible stumbling block is probably less of a problem for scalable vectors, e.g. What do you think? Are we worrying unnecessarily? |
|
That's fine. |
Thank you for pointing this out @AssadHashmi ! |
New record_filter_t::record_filter_func_t filter, which we call "encodings2regdeps", that modifies the encoding of trace_entry_t records from a real ISA to the synthetic regdeps ISA. "encodings2regdeps" can add or remove trace_entry_t records that contain encodings depending on the regdeps encoding size. Note that "encodings2regdeps" only changes the encoding of instructions, but it does not adjust their length (or changes the instruction PC), hence the output trace will have encoding sizes that do not match the instruction length. For this reason we disable the encoding size vs instruction length check in reader_t when the trace has DR_ISA_REGDEPS encodings. This filter is part of the "record_filter" tool and can be invoked with: ``` drrun -t drcachesim -simulator_type record_filter -filter_encodings2regdeps -indir path/to/input/trace -outdir path/to/output/trace ``` Issue: #6662
Fixes a data race due to multiple dr_standalone_init() done in parallel (per shard) by encodings2regdeps_filter_t. dcontext is now initialize one time by record_filter_t and passed to its filters through the record_filter_info_t interface. Issue #6662
Fixes a data race due to multiple dr_standalone_init() done in parallel (per shard) by encodings2regdeps_filter_t. dcontext is now initialized one time by record_filter_t and passed to its filters through the record_filter_info_t interface. Avoids a data race in opcode_mix where all threads set the dcontext isa_mode to DR_ISA_REGDEPS for regdeps input traces. Issue #6662 #6812
Adds disassembling for DR_ISA_REGDEPS instructions. Specifically, when we disassemble DR_ISA_REGDEPS instructions, we print the instruction encoding (divided in 4 byte words, can span one or two lines, similar to x86), we substitute the opcode with categories, we print the operations size (e.g., `[4byte]`), and then the source and destination virtual register names (e.g., `%rv3`). Disassembled instructions look as follows: `00000812 06260606 load [8byte] %rv4 -> %rv4 %rv36` In general, they follow this pattern: [encoding (in 4 byte words)] [categories] [operation_size] [src_regs -> dst_regs] Issue: #6662
Modifies the view tool to handle OFFLINE_FILE_TYPE_ARCH_REGDEPS traces, leveraging the disassembly of DR_ISA_REGDEPS instructions. When visualizing DR_ISA_REGDEPS instructions, the view tool still prints the instruction length and PC, which for OFFLINE_FILE_TYPE_ARCH_REGDEPS traces are the same as those in the original trace. Then, after the PC, the instruction encoding, categories, operation size, and registers are printed following the disassembly format of DR_ISA_REGDEPS instructions (xref: #6799). DR_ISA_REGDEPS instructions printed by the view tool look as follows: ``` [...] ifetch 10 byte(s) @ 0x00007f86ef03d107 00001931 04020204 load store [4byte] %rv0 %rv2 %rv36 -> %rv0 [...] 00000026 ``` We also fix a formatting bug in DR_ISA_REGDEPS instruction disassembly, where we were missing a new line when the instruction encoding spills into a second line. Issue: #6662
The public version of a trace needs to preserve the TRACE_MARKER_TYPE_FUNC_[ID | ARG | RETVAL] markers related to SYS_futex system calls. We add this feature to encodings2regdeps_filter_t. This filter still drops the TRACE_MARKER_TYPE_FUNC_ markers related to other functions that are not SYS_futex. However, we still rely on type_filter_t to remove the additional markers that we don't want to preserve in the public trace. Issue #6662
We want to create a new tool to filter traces of Google workloads for public release.
The new public Google workload traces will contain more information compared to the previous version, while still preserving confidentiality of Google's IP.
Main features of new public traces:
The text was updated successfully, but these errors were encountered: