Build report would not detect a nested FV if the nested
FV was not in a subsection of an FFS statement.
Modify the build report to better handle some of the
variations of nested FVs.
Failing Example:
[Fv.FvName1]
INF <path to some driver>.inf
[Fv.FvName0]
FILE FV_IMAGE = B25ACDEF-39CE-4FA5-B50A-33E24DB1BDDF {
SECTION FV_IMAGE = FvName1
}
Working Example:
[Fv.FvName1]
INF <path to some driver>.inf
[Fv.FvName0]
FILE FV_IMAGE = DA04F6BF-A0FD-47EC-928B-5101A6C95026 {
SECTION GUIDED EE4E5898-3914-4259-9D6E-DC7BD79403CF
PROCESSING_REQUIRED = TRUE {
SECTION FV_IMAGE = FvName1
}
}
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
Add GUID section for build report.
Also, change the GUID format to string format to be easier to parse
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
If an FV_SPARE_SPACE_THRESHOLD is enabled and an FV
is 100% full with 0 bytes free, then this is likely
a special FV that may have alignment requirements
for the FFS file for both the start address and the
length and the FFS file consumes all the available
FV space.
Reduce FV_SPARE_SPACE_THRESHOLD from an error to a
warning if this FV 0 bytes free condition is
detected.
PR #10828 introduced the generation of these error
conditions for an FV with large alignment requirements.
The pad region before the aligned FFS file used to
be counted as free space even though it could never
be used due to the alignment requirements. There was
actually no free space available. PR #10828 fixed the
free space calculation to properly show it as 0 bytes
free, and this change then caused build error when
FV_SPARE_SPACE_THRESHOLD feature was enabled. The
reduction to a warning for this condition allows the
build to complete with errors and also provides a
build log warning message for review.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Refer to the docs of python, `os.path.normcase(path)` function:
"Normalize the case of a pathname. On Windows, convert all characters in
the pathname to lowercase, and also convert forward slashes to backward
slashes. On other operating systems, return the path unchanged."
`os.path.normpath(path)` also convert forward slashes to backward slashes.
So call `os.path.normcase` after `os.path.normpath` just convert path to
lowercase on Windows(only).
And Windows is case-insensitive but case-preserving.
So the usage of `os.path.normcase(os.path.normpath(path))` can be
simplified to `os.path.normpath(path)`. Then we can use case-preserving
paths rather than lowercase paths in compile_commands.json file
or build log.
But this patch continue to use `os.path.normcase`
when comparing/searching paths.
Signed-off-by: Yang Gang <yanggang@byosoft.com.cn>
This patch updates the GenC logic to generate a random stack cookie value
for the stack check libraries. These random values improve security
for modules which cannot update the global intrinsics.
If the stack cookie value is randomized in the AutoGen.h file each
build, the build system will determine the module/library must be
rebuilt causing effectively a clean build every time. This also makes
binary reproducibility impossible.
This patch updates the early build scripts to create 32 and 64-bit JSON
files in the build output directory which each contain 100 randomized
stack cookie values for each bitwidth. If the JSON files are already
present, then they are not recreated which allows them to be stored and
moved to other builds for binary reproducibility. Because they are in
the build directory, a clean build will cause the values to be
regenerated.
The logic which creates AutoGen.h will read these JSON files and use a
hash of the module GUID (the hash seed is fixed in Basetools) to index
into the array of stack cookie values for the module bitwidth. This
model is necessary because there isn't thread-consistent data so we
cannot use a locking mechanism to ensure only one thread is writing to
the stack cookie files at a time. With this model, the build threads
only need to read from the files.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
This produces output that matches CodeChecker log command
- Set directory to build output path
- Set build destination to the object created instead of the path
- Add recursive macro support
- Add lookup in module.Macros dictionary
- Add leading include flag to include list
- Add source file to compile commands
Signed-off-by: Jeff Brasen <jbrasen@nvidia.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2850
Add "-Y REPORT_INFO" option to build command to generate compile
information as part of BuildReport.
This option generates files to be used by external tools as IDE's
to enhance functionality.
Files are created inside build folder:
<Build>/<BuildTarget>/<ToolChain>/CompileInfo
Files created:
* compile_commands.json - Compilation Database. To be used by IDE's
to enable advance features
* cscope.files - List of files used in compilation. Used by Cscope to parse
C code and provide browse functionality.
* module_report.json - Module data form buildReport in Json format.
Signed-off-by: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653
The "target.txt" and "tools_def.txt" filenames are hard-coded
at some places when global definitions are available at:
BaseTools/Source/Python/Common/TargetTxtClassObject.py:
DefaultTargetTxtFile
and
BaseTools/Source/Python/Common/ToolDefClassObject.py:
DefaultToolsDefFile
Use these global definitions instead.
Also remove the unused gBuildConfiguration and gToolsDefinition
variables from build.py
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3653
Running the following command:
python3 build/build.py -a AARCH64 -t GCC5
-p ArmPlatformPkg/ArmPlatformPkg.dsc -b DEBUG libraries
triggers the following error:
make: *** Build/ArmPlatform/DEBUG_GCC5/AARCH64/MdePkg/Library/
BasePcdLibNull/BasePcdLibNull: Is a directory. Stop.
Indeed, MakefileName is set to en empty string. Setting MakefileName
resolves the error.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Reviewed-by: Chris Jones <christopher.jones@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
threading camelCase functions have preferred alternatives since
python2.6. python3.10 has started emitting DeprecationWarnings
for them
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3359
* Save/Restore global state in GetToolChainAndFamilyFromDsc()
This resolves an issue where the multi-arch build for
UefiPayloadPkg would skip the autogen and build of IA32
components.
* Expand tools wildcard.
This resolves the issue where autogen makefile contents
would have a BUIDLRULEFAMILY tools definitions with an
'*' in the tool field that breaks the build from invalid
makefile syntax.
* Build rule family higher priority than Family.
This resolves the issue where flags were appended from
both the BUILDRULEFAMILY and FAMILY when only
BUILDRULEFAMILY should be appended when present.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Andrew Fish <afish@apple.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3359
Update BaseTools to support new build targets, new tool chains,
and new architectures declared in DSC file [BuildOptions] sections.
* Do not expand * when tools_def.txt is parsed. Only expand when
both tools_def.txt and DSC [BuilsOptions] sections have been parsed.
This also requires more flexible matching of tool keys that contain *
in tool key fields.
* Pre-scan the platform DSC file for FAMILY and TOOLCHAIN declarations
DSC in [BuildOptions] sections before the FAMILY and TOOLCHAIN need
to be known.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
In the following corner case, the build report
will crash. This patch is to fix this problem.
Case:
Multiple SKU are used and 2 more DynamicHii structure Pcds
are set in dsc file under different SKU. And 1 more of those
Pcds are not used in any INF file.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
AutoGen manager/workers halt the progress when an error occurs but
doesn't propagate the error code to main and allows main exit with 0
and gets the build system unable to catch the occurrence of an error.
This change informs main with an error when a progress is halted and
helps main exit with 1.
Signed-off-by: Irene Park <ipark@nvidia.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654
If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.
This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=10000.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538
For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.
*_*_*_LZMA_PATH = LzmaCompressPlatform
This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.
Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make
This prevents using the GCC compiler toolchain on a
Windows build host.
To address this issue this patch introduces 2 factors
to determine the generated makefile output.
1. Platform -> to determine shell commands used
in makefile.
2. MakeTool -> to determine the type of makefile
that needs to be generated.
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2455
BuildOption is used by TargetTxtClassObj.py
GenFdsOption is used by GenFds.py
When the GenFds tool is used alone (e.g. python3 -m GenFds.GenFds -h)
With the OptionParser function, the first detected function
prints the help message
import TargetTxtClassObj to GenFds,
The BuildOption will be executed and replace GenFdsOption
We removed all objects associated with this problem that
were created directly during the import process
(e.g. BuildOption, BuildTarget = MyOptionParser(),
TargetTxt = TargetTxtDict())
The Patch is going to fix this issue
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
In patch set 13c5e34a - 0c3e8e99, we implemented incremental build with
using compiler/pre-processor generate dependent header file function.
A issue is found for MSVC compiler, that the cl.exe /showIncludes
build option generate header file list to either stdout or stderr.
For .c file, the header file list is print out to stdout while for
.vfr, .aslc and .nasm file, the file list is print out to stderr.
The build tool use two threads to process the message from stdout and
stderr, but to generate correct *.deps file, build tool need to
combine the header file list from stderr and other messages from stdout
together with correct time sequence order.
So this patch is trying to combine the stdout and stderr together for
the process which is for calling make program.
The impact of this patch is that the output message of build with -q
will be changed. The compiler error message will not print out.
The build behavior of other log level setting will not be impacted.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2371
This patch is to fix a regression issue that build fails
if multiple build targets given.
Two changes cause this regression issue.
One is AutoGen object __hash__ function only
hash file path and arch, missing ToolChain and build target.
The other is changing the multiple-thread-genfds function as default
build behavior. To generate the genffs command to Makefile, there
is a global data set is used, GenFdsGlobalVariable, which cause build
tool use the data of first build-target build in
the second build-target build.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Redesign the binary cache and bases on the compiler to
output the dependency header files info for every module.
The binary cache will directly consume the dependency header
files info and doesn't parse the C source code by iteself.
Also redesign the dependency files list format for module
and try to share the common lib hash result as more as
possible in local process. Remove the unnecessary share data
access across multiprocessing.
Signed-off-by: Steven Shi <steven.shi@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2363
This patch is to fix a build tool regression issue which was introduced
by commit b8ac0b7f28.This issue caused map file lost the line of IMAGE=***.
For example,in Ovmf.map, there is no line of (IMAGE=<path to efi> ) under
each of modules item.
The path to the efi file generated by each module is written on this line
The purpose of this line is add the debug image full path.
there is no information about the module in the map file other than FVName,
it allows us to quickly know which module this part corresponds to.
In commit b8ac0b7f28,add a line ("self.BuildModules = []") in function,
but it's used to calculate the variable ModuleList in the following code.
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2201
An error occurs using special VpdPcd that is not used in the Inf file
In dsc:
[PcdsDynamicExVpd.common.DEFAULT]
gBoardModuleTokenSpaceGuid.test1|*|{CODE({
{0x0} // terminator
})}
In dec:
[PcdsDynamicEx]
# Vpd GPIO table
gBoardModuleTokenSpaceGuid.test1|{0}|GPIO_INIT_CONFIG[]|0x50000018 {
<HeaderFiles>
Library/GpioLib.h
<Packages>
MdePkg/MdePkg.dec
}
ValueError: invalid literal for int() with base 0: '*'
This Patch is going to fix issue
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2188
build -p MdeModulePkg\MdeModulePkg.dsc -a IA32 -m
MdeModulePkg\Universal\PCD\Pei\Pcd.inf
Error:
AttributeError: 'PlatformInfo' object has no attribute
'DynamicPcdList'
The DSC data object used to build a separate module today
is PlatformInfo rather than PlatformAutoGen
'PlatformAutoGen' object has attribute 'DynamicPcdList'
This patch is going to fixed this issue
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
