Adds support for building the C language BaseTools for Windows using
toolchains based on mingw-w64.
Mingw-w64 is a collection of header files, libraries, and tools that
when combined with a compiler enable development of Windows software.
Mingw-w64 is a fork of the original MinGW (Minimalist GNU for Windows).
Most active development on MinGW has ceased and mingw-w64 is now the
actively maintained successor. Mingw-w64 provides a libc implementation
built on top of Microsoft's UCRT (Universal C Runtime) with all
nessesary compiler bindings needed to support the C++11 feature set.
Modern mingw-w64 development appears to have coalesced around MSYS2,
which produces a distributions of both GCC and LLVM/Clang that use
mingw-w64 to target the Windows OS. This MSYS2 Clang distribution has a
UNIX-like directory layout and includes Windows binaries of GNU Make.
Combined with the open source licensing, MSYS2's Clang distribution is a
highly attractive choice as an alternative Windows SDK for open source
projects such as TianoCore.
If one wishes to use EDK II to build UEFI firmware on the Windows
platform, then the C BaseTools need to be compiled as Windows
applications. This includes the PcdValueInit.exe program, which needs
to be recompiled every time a firmware build is run in order to
regenerate the initial values for structured PCDs. Currently, BaseTools
only supports the Visual C++ toolchain on the Windows platform. The
following new features have been added to enable usage of the toolchains
derived from mingw-w64:
- Fixes to the BaseTools C source code to support the use of a
GCC-style compiler on the Windows OS.
- The GNU Make-style Makefiles for the C BaseTools have been modified
to support Windows. Both GCC + mingw-w64 and Clang + mingw-w64 have
been tested and confirmed to build a working BaseTools.
- BaseTools now supports generating GNU Make-style Makefiles on the
Windows platform for the purpose of building firmware.
- edksetup.bat has been modified to optionally build BaseTools via
mingw-w64. There is no impact to the existing support for Visual C++
and Visual C++ remains the default toolchain.
Usage Instructions:
For the vast majority of users, the only system setup change nessesary
to use a mingw-w64 toolchain is to set the BASETOOLS_MINGW_PATH to the
directory containing the desired mingw-w64 based toolchain.
A new command line argument has been added to edksetup.bat: Mingw-w64
If this command line argument is set, then the script will set the
BASETOOLS_MINGW_BUILD environment variable. The user can also opt to set
this environment variable manually before running edksetup.bat
If BASETOOLS_MINGW_BUILD is defined, then the BASETOOLS_MINGW_PATH
environment variable must point to the directory containing the
mingw-w64 toolchain.
If CLANG_BIN is not defined and %BASETOOLS_MINGW_PATH%\bin\clang.exe
exists, then edksetup.bat will set CLANG_BIN=%BASETOOLS_MINGW_PATH%\bin\
This removes the requirement to configure the CLANG_BIN environment
variable manually in order to run a CLANGPDB or CLANGDWARF build if one
has the MSYS2 Clang distribution installed. If one wishes to use a
different copy of Clang (for example official LLVM binaries) to build
firmware and only use the MSYS2 Clang to build BaseTools, then one can
continue to set the CLANG_BIN environment variable, same as before. I
have tested the MSYS2 Clang distribution against the official LLVM
distribution and can confirm that if the compiler version is the same
the emitted machine code is identical between the two. Interestingly,
the MSYS2 Clang distribution emits the path to the PDB file using "/" as
the path seperator instead of "\". That appears to be the only
difference in output. Therefore, using the MSYS2 Clang distribution to
compile firmware seems a reasonable choice.
If CLANG_HOST_BIN is not defined and BASETOOLS_MINGW_BUILD is defined
and %BASETOOLS_MINGW_PATH%\bin\mingw32-make.exe exists, then
edksetup.bat will add %BASETOOLS_MINGW_PATH%\bin\ to the PATH and set
CLANG_HOST_BIN=mingw32-
This enable usage of the GNU Make included in the mingw-w64 toolchain
to build firmware in addition to BaseTools. if BASETOOLS_MINGW_BUILD is
not defined, edksetup.bat will continue to set CLANG_HOST_BIN=n, which
uses nmake to build firmware. This behavior can be overridden by
manually setting the value of CLANG_HOST_BIN before executing
edksetup.bat if one wishes to use a specific Make utility for the
CLANGPDB/CLANGDWARF toolchains.
References:
- https://www.mingw-w64.org/
- https://www.msys2.org/
Co-authored-by: Sandesh Jain <sandesh.jain@intel.com>
Signed-off-by: Nate DeSimone <nathaniel.l.desimone@intel.com>
Fixes a BaseTools Bin directory path detection bug in the the BaseTools
Unit Tests. The script incorrectly assumes that sys.platform
will be win64 on a 64-bit Python interperter.
The "win64" platform string has not been used for 64-bit Python
interperters since May 10, 2000:
https://github.com/python/cpython/commit/da5cc82d
Moreover, this patch was merged before the Python 2.0 release, so there
never has been a released Python interperter that used the "win64"
string.
Signed-off-by: Nate DeSimone <nathaniel.l.desimone@intel.com>
On Windows, executables have a '.exe' suffix which needs to be added for
them to be found in a path.
Also, files need to be explicitly opened as binary.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Update toolsetup.bat and Tests/PythonTest.py to check if we're running a
version of Python that's compatible with BaseTools and the Pip
BaseTools.
BaseTools uses syntax from Python 3.6 or newer, so set that as the minimum
version EDK2 requires.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Commit ("2355f0c09c52 BaseTools: Fix check for ${PYTHON_COMMAND} in
Tests/GNUmakefile") fixed a latent issue in the BaseTools/Tests
Makefile, but inadvertently broke the BaseTools build for cases where
PYTHON_COMMAND is not set. As it turns out, running 'command' without a
command argument makes the invocation succeed, causing the empty
variable to be evaluated and called later.
Let's put double quotes around PYTHON_COMMAND in the invocation of
'command' and force it to fail when PYTHON_COMMAND is not set.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
When checking if $PYTHON_COMMAND exists, curly braces should
be used instead of parentheses.
Also, "1" causes an error on FreeBSD: it's likely supposed to
be 2>&1 like other scripts.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
This reverts commit df7c81b5b2.
In commint df7c81b5b219c9,used a function to test python module
But the Boolean value of the return value of this function is the
opposite of the Correct result, resulting in an unexpected result
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=1390
1. support hex number for array index
2. support Non-Dynamic Pcd for array data type
3. support {} and {CODE()} for array data type
4. Change GetStructurePcdMaxSize to be a static function since it need to
be called in another static function. And this function does not depend on
it's class instance.
5. Add unittest for RemoveCComments function and
ArrayIndex regular expression.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Philippe Mathieu-Daud? <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This reverts commit 6693f359b3c213513c5096a06c6f67244a44dc52..
678f851312.
Python3 migration is the fundamental change. It requires every developer
to install Python3. Before this migration, the well communication and wide
verification must be done. But now, most people is not aware of this change,
and not try it. So, Python3 migration is reverted and be moved to edk2-staging
Python3 branch for the edk2 user evaluation.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
1. Do not use tab characters
2. No trailing white space in one line
3. All files must end with CRLF
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
"types.TypeType" is now an alias of the built-in "type" and is not
compatible with python 3.
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Refactor print statements to be compatible with python 3.
Based on "futurize -f libfuturize.fixes.fix_print_with_import"
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Convert "except ... ," to "except ... as" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_except"
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Surrogate pair characters can be encoded in UTF-8 files, but they are
not valid UCS-2 characters.
For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\ud801', 'utf-8')
'\xed\xa0\x81'
But, the range of 0xd800 - 0xdfff should be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.
We test that this case is rejected for UTF-8 with and without the
UTF-8 BOM.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17698 6f19259b-4bc3-4df7-8a09-765794883524
Since UTF-8 .uni unicode files might contain strings with unicode code
points larger than 16-bits, and UEFI only supports UCS-2 characters,
we need to make sure that BaseTools rejects these characters in UTF-8
.uni source files.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17697 6f19259b-4bc3-4df7-8a09-765794883524
Supplementary Plane characters can exist in UTF-16 files,
but they are not valid UCS-2 characters.
For example, this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\U00010300', 'utf-16')
'\xff\xfe\x00\xd8\x00\xdf'
Therefore the UCS-4 0x00010300 character is encoded as two
16-bit numbers (0xd800 0xdf00) in a little endian UTF-16
file.
For more information, see:
http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF
This test checks to make sure that BaseTools will reject these
characters in UTF-16 files.
The range of 0xd800 - 0xdfff should also be rejected as unicode code
points because they are reserved for the surrogate pair usage in
UTF-16 files.
This test was fixed by the previous commit:
"BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni files"
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17695 6f19259b-4bc3-4df7-8a09-765794883524
This allows unit tests to easily include BaseTools python
modules. This is very useful for writing unit tests.
Actually, previously, we would do this when RunTests.py was executed,
so unit tests could easily import BaseTools modules, so long as they
were executed via RunTests.
This change allows running the unit test files individually which can
be faster for developing the new unit test cases.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17691 6f19259b-4bc3-4df7-8a09-765794883524
