Trigger to generate Windows boot files

Posted by Yuriy on Friday, December 04, 2020

Cobbler Windows installation

Many companies now have a separate infrastructure for deploying Linux servers and Windows workstations, which leads to additional costs. This kind of overhead can be reduced by using a single platform to install both kinds of operating systems.

Despite the success of Microsoft in creating tools for integrating with Linux, in my opinion, Linux is more suitable for such a platform thanks to the wonderful project Cobbler. Cobbler is incredibly flexible and can be tailored to suit any need. Using templates and Cobbler profiles with the ability to inherit (parent profile), you can change a couple of lines in the template, you can very quickly create an installation option adapted to the needs of a particular user without having to depend on the capabilities of the closed proprietary code.

Cobbler has an excellent architectural feature of expanding functionality by writing triggers that do not affect the main code. We used this opportunity to prepare the necessary files for a Windows installation.

Linux currently has all the necessary tools to prepare these files. All that remains is to combine these tools in the trigger code to get an open source infrastructure for deploying Windows without using proprietary Microsoft tools.

Trigger sync_post_wingen.py:

some of the files are created from standard ones (pxeboot.n12, bootmgr.exe) by directly replacing one string with another directly in the binary
in the process of changing the bootmgr.exe file, the checksum of the PE file will change and it needs to be recalculated. The trigger does this with python-pefile
python3-hivex is used to modify Windows boot configuration data (BCD). For pxelinux distro boot_loader in BCD, paths to winpe.wim and boot.sdi are generated as /winos/<distro_name>/boot, and for iPXE - \Boot.
uses wimlib-tools to replace startnet.cmd startup script in WIM image

Windows answer files (autounattended.xml) are generated using Cobbler templates, with all of its conditional code generation capabilities, depending on the Windows version, architecture (32 or 64 bit), installation profile, etc.

Startup scripts for WIM images (startnet.cmd) and a script that is launched after OS installation (post_install.cmd) are also generated from templates

Post-installation actions such as installing additional software, etc., are performed using the Automatic Installation Template (win.ks).

A logically automatic network installation of Windows 7 and newer can be represented as follows:

PXE + Legacy BIOS Boot

Original files: pxeboot.n12 → bootmgr.exe → BCD → winpe.wim → startnet.cmd → autounattended.xml
Cobbler profile 1: pxeboot.001 → boot001.exe → 001 → wi001.wim → startnet.cmd → autounatten001.xml → post_install.cmd profile_name
...

iPXE + UEFI BIOS Boot

Original files: ipxe-x86_64.efi → wimboot → bootx64.efi → BCD → winpe.wim → startnet.cmd → autounattended.xml
Cobbler profile 1: ipxe-x86_64.efi → wimboot → bootx64.efi → 001 → wi001.wim → startnet.cmd → autounatten001.xml → post_install.cmd profile_name
...

For older versions (Windows XP, 2003):

Original files: pxeboot.n12 → setupldr.exe → winnt.sif → post_install.cmd profile_name
Cobbler profile <xxx>: pxeboot.<xxx> → setup<xxx>.exe → wi<xxx>.sif → post_install.cmd profile_name

Preparing for an unattended network installation of Windows

dnf install python3-pefile python3-hivex wimlib-utils
In the server’s tftp directory, create a directory winos

mkdir /var/lib/tftpboot/winos

and copy the Windows distributions there:

dr-xr-xr-x. 1 root   root         200 Mar 23  2017 Win10_EN-x64
dr-xr-xr-x. 1 root   root         238 Aug  7  2015 Win2012-Server_EN-x64
dr-xr-xr-x. 1 root   root         220 May 17  2019 Win2016-Server_EN-x64
drwxr-xr-x. 1 root   root         236 Dec  3 22:42 Win2019-Server_EN-x64
dr-xr-xr-x. 1 root   root         788 Aug  8  2015 Win2k3-Server_EN-x64
dr-xr-xr-x. 1 root   root         196 Sep 24  2017 Win2k8-Server_EN-x64
dr-xr-xr-x. 1 root   root         132 Aug  8  2015 Win7_EN-x64
dr-xr-xr-x. 1 root   root         238 Aug  7  2015 Win8_EN-x64
dr-xr-xr-x. 1 root   root         456 Aug  8  2015 WinXp_EN-i386

Copy the following files to the distributions directories (for Windows 7 and newer):

PXE + Legacy BIOS Boot

pxeboot.n12
bootmgr.exe
boot/BCD
boot/boot.sdi

iPXE + UEFI BIOS Boot

ipxe-x86_64.efi
wimboot
boot/bootx64.efi
boot/BCD
boot/boot.sdi

Share /var/lib/tftpboot/winos via Samba:

  vi /etc/samba/smb.conf
          [WINOS]
          path = /var/lib/tftpboot/winos
          guest ok = yes
          browseable = yes
          public = yes
          writeable = no
          printable = no

You can use tftpd.rules to indicate the actual locations of the bootmgr.exe and BCD files generated by the trigger

cp /usr/lib/systemd/system/tftp.service /etc/systemd/system

Replace the line in the /etc/systemd/system/tftp.service

ExecStart=/usr/sbin/in.tftpd -s /var/lib/tftpboot
    to:
ExecStart=/usr/sbin/in.tftpd -m /etc/tftpd.rules -s /var/lib/tftpboot

Create a file /etc/tftpd.rules:

vi /etc/tftpd.rules
rg	\\					/ # Convert backslashes to slashes
r	(BOOTFONT\.BIN)			/winos/\1
r	(/Boot/Fonts/)(.*)			/winos/Fonts/\2

r	(ntdetect\....)			/winos/\1

r	(wine.\.sif)				/WinXp_EN-i386/\1
r	(xple.)					/WinXp_EN-i386/\1
r	(/WinXp...-i386/)(.*)			/winos\1\L\2

r	(wi2k.\.sif)				/Win2k3-Server_EN-x64/\1
r	(w2k3.)					/Win2K3-Server_EN-x64/\1
r	(/Win2k3-Server_EN-x64/)(.*)		/winos\1\L\2

r	(boot7e.\.exe)				/winos/Win7_EN-x64/\1
r	(/Boot/)(7E.)				/winos/Win7_EN-x64/boot/\2

r	(boot28.\.exe)				/winos/Win2k8-Server_EN-x64/\1
r	(/Boot/)(28.)				/winos/Win2k8-Server_EN-x64/boot/\2

r	(boot9r.\.exe)				/winos/Win2019-Server_EN-x64/\1
r	(/Boot/)(9r.)				/winos/Win2019-Server_EN-x64/boot/\2

r	(boot6r.\.exe)				/winos/Win2016-Server_EN-x64/\1
r	(/Boot/)(6r.)				/winos/Win2016-Server_EN-x64/boot/\2

r	(boot2e.\.exe)				/winos/Win2012-Server_EN-x64/\1
r	(/Boot/)(2e.)				/winos/Win2012-Server_EN-x64/boot/\2

r	(boot81.\.exe)				/winos/Win8_EN-x64/\1
r	(/Boot/)(B8.)				/winos/Win8_EN-x64/boot/\2

r	(boot1e.\.exe)				/winos/Win10_EN-x64/\1
r	(/Boot/)(1E.)				/winos/Win10_EN-x64/boot/\2

Add information about Windows distributions to the distro_signatures.json file

"windows": {
 "2003": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub"]}
 },
 "2008": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "2012": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "2016": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "2019": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "XP": {
  "supported_arches":["i386","x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub"]}
 },
 "7": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "8": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 },
 "10": {
  "supported_arches":["x86_64"],
  "boot_loaders":{"x86_64":["pxelinux","grub","ipxe"]}
 }
},

Add trigger /usr/lib/python3.9/site-packages/cobbler/modules/sync_post_wingen.py

Cobbler Windows Templates

/var/lib/tftpboot/winos/startnet.template is used to generate /Windows/System32/startnet.cmd script in WIM image

Example:

    wpeinit
    
    ping 127.0.0.1 -n 10 >nul
    md \tmp
    cd \tmp
    ipconfig /all | find "DHCP Server" > dhcp
    ipconfig /all | find "IPv4 Address" > ipaddr
    FOR /F "eol=- tokens=2 delims=:" %%i in (dhcp) do set dhcp=%%i
    FOR  %%i in (%dhcp%) do set dhcp=%%i
    FOR /F "eol=- tokens=2 delims=:(" %%i in (ipaddr) do set ipaddr=%%i
    
    net use y: \\@@http_server@@\Public /user:install install
    #set $distro_dir = '\\\\' + $http_server + '\\WINOS\\' + $distro_name
    net use z: $distro_dir /user:install install
    set exit_code=%ERRORLEVEL%
    IF %exit_code% EQU 0 GOTO GETNAME
    echo "Can't mount network drive"
    goto EXIT
    
    :GETNAME
    y:\windows\bind\nslookup.exe %ipaddr% | find "name =" > wsname
    for /f "eol=- tokens=2 delims==" %%i in (wsname) do echo %%i > ws
    for /f "eol=- tokens=1 delims=." %%i in (ws) do set wsname=%%i
    FOR  %%i in (%wsname%) do set wsname=%%i
    
    #set $unattended = "set UNATTENDED_ORIG=Z:\\sources\\" + $kernel_options["sif"]
    $unattended
    set UNATTENDED=X:\tmp\autounattended.xml
    
    echo off
    FOR /F "tokens=1 delims=!" %%l in (%UNATTENDED_ORIG%) do (
       IF "%%l"=="            <ComputerName>*</ComputerName>" (
         echo             ^<ComputerName^>%wsname%^<^/ComputerName^>>> %UNATTENDED%
       ) else (
         echo %%l>> %UNATTENDED%
       )
    )
    echo on
    
    :INSTALL
    set n=0
    z:\sources\setup.exe /unattend:%UNATTENDED%
    set /a n=n+1
    ping 127.0.0.1 -n 5 >nul
    IF %n% lss 20 goto INSTALL
    
    :EXIT

Templates /var/lib/tftpboot/winos/{winpe7,winpe8}.template are standard or customized WIM PE images. The trigger copies to the directory of the corresponding distro and changes the contents of startnet.cmd based on the corresponding template and Cobbler profile. winpe7 is used for Windows 7 and Windows 2008 Server, and winpe8 for newer versions
/var/lib/tftpboot/winos/win_sif.template is used to generate /var/lib/tftpboot/winos/<distro_name>/sources/autounattended.xml in case of Windows 7 and newer or winnt.sif for Windows XP, 2003

Example:

    #if $arch == 'x86_64'
            #set $win_arch = 'amd64'
    #else if $arch == 'i386'
            #set $win_arch = 'i386'
    #end if
    
    #set $OriSrc = '\\\\' + $http_server + '\\WINOS\\' + $distro_name + '\\' + $win_arch
    #set $DevSrc = '\\Device\\LanmanRedirector\\' + $http_server + '\\WINOS\\' + $distro_name
    
    #if $distro_name in ( 'WinXp_EN-i386', 'Win2k3-Server_EN-x64' )
    [Data]
    floppyless = "1"
    msdosinitiated = "1"
    ; Needed for second stage
    OriSrc="$OriSrc"
    OriTyp="4"
    LocalSourceOnCD=1
    DisableAdminAccountOnDomainJoin=0
    AutomaticUpdates="No"
    Autopartition="0"
    UnattendedInstall="Yes"
    <..>
    [GuiRunOnce]
    "%Systemdrive%\post_install.cmd @@profile_name@@"
    <..>
    #else if $distro_name in ('Win7_EN-x64', 'Win2k8-Server_EN-x64', 'Win2012-Server_EN-x64', 'Win2016-Server_EN-x64', 'Win2019-Server_EN-x64', 'Win8_EN-x64', 'Win10_EN-x64' )
    <?xml version="1.0" encoding="utf-8"?>
    <unattend xmlns="urn:schemas-microsoft-com:unattend">
    #if $distro_name in ( 'Win2012-Server_EN-x64' )
        <servicing>
            <package action="configure">
    <..>
                </DiskConfiguration>
                <ImageInstall>
                    <OSImage>
                        <InstallFrom>
                            <Credentials>
                                <Domain></Domain>
                            </Credentials>
                            <MetaData wcm:action="add">
                                <Key>/IMAGE/NAME</Key>
    #else if $distro_name in ( 'Win7_EN-x64' )
                                <Value>Windows 7 PROFESSIONAL</Value>
    #else if $distro_name in ( 'Win2k8-Server_EN-x64' )
                                <Value>Windows Server 2008 R2 SERVERENTERPRISE</Value>
    <..>
            <component name="Microsoft-Windows-PnpCustomizationsWinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                <DriverPaths>
    #if $distro_name in ( 'Win2012-Server_EN-x64', 'Win8_EN-x64' )
                    <PathAndCredentials wcm:action="add" wcm:keyValue="1">
                        <Path>\\@@http_server@@\WINOS\Drivers\CHIPSET\Win8</Path>
                    </PathAndCredentials>
    <..>
                <FirstLogonCommands>
                    <SynchronousCommand wcm:action="add">
                        <RequiresUserInput>false</RequiresUserInput>
                        <Order>1</Order>
                        <CommandLine>c:\post_install.cmd @@profile_name@@</CommandLine>
                    </SynchronousCommand>
                </FirstLogonCommands>
    <..>

The post_inst_cmd.template is used to generate a script that is launched after OS installation in the `autounattended.xml` section, or [GuiRunOnce] in `winnt.sif`

Example:

    %systemdrive%
    CD %systemdrive%\TMP >nul 2>&1
    $SNIPPET('my/win_wait_network_online')
    wget.exe http://@@http_server@@/cblr/svc/op/ks/profile/%1
    MOVE %1 install.cmd
    todos.exe install.cmd
    start /wait install.cmd
    DEL /F /Q libeay32.dll >nul 2>&1
    DEL /F /Q libiconv2.dll >nul 2>&1
    DEL /F /Q libintl3.dll >nul 2>&1
    DEL /F /Q libssl32.dll >nul 2>&1
    DEL /F /Q wget.exe >nul 2>&1
    DEL /F /Q %0 >nul 2>&1

For the script to work, you need to place the following files in the /var/lib/tftpboot/winos/<distro_name>/$OEM$/$1/TMP directory:

ls -l '/var/lib/tftpboot/winos/Win10_EN-x64/$OEM$/$1/TMP'
total 2972
-rwxr-xr-x. 1 root root 1177600 Sep  4  2008 libeay32.dll
-rwxr-xr-x. 1 root root 1008128 Mar 15  2008 libiconv2.dll
-rwxr-xr-x. 1 root root  103424 May  6  2005 libintl3.dll
-rwxr-xr-x. 1 root root  232960 Sep  4  2008 libssl32.dll
-rwxr-xr-x. 1 root root    4880 Oct 26  1999 sleep.exe
-rwxr-xr-x. 1 root root   52736 Oct 27  2013 todos.exe
-rwxr-xr-x. 1 root root  449024 Dec 31  2008 wget.exe

The win_wait_network_online snippet might look something like this:

:wno10
set n=0

:wno20
ping @@http_server@@ -n 3
set exit_code=%ERRORLEVEL%

IF %exit_code% EQU 0 GOTO wno_exit
set /a n=n+1
IF %n% lss 30 goto wno20
pause
goto wno10

:wno_exit

win.ks - Automatic Installation Template, which is specified for the Cobbler profile in cobbler profile add/edit --autoinstall=win.ks .. command

Example:

    $SNIPPET('my/win_wait_network_online')
    
    set n=0
    
    :mount_y
    net use y: \\@@http_server@@\Public /user:install install
    set exit_code=%ERRORLEVEL%
    
    IF %exit_code% EQU 0 GOTO mount_z
    set /a n=n+1
    IF %n% lss 20 goto mount_y
    PAUSE
    goto mount_y
    
    set n=0
    
    :mount_z
    net use z: \\@@http_server@@\winos /user:install install
    set exit_code=%ERRORLEVEL%
    
    IF %exit_code% EQU 0 GOTO mount_exit
    set /a n=n+1
    IF %n% lss 20 goto mount_z
    PAUSE
    goto mount_z
    
    :mount_exit
    if exist %systemdrive%\TMP\stage.dat goto flag005
    echo 0 > %systemdrive%\TMP\stage.dat
    
    $SNIPPET('my/win_check_virt')
    
    #if $distro_name in ( 'WinXp_EN-i386', 'Win2k3-Server_EN-x64' )
    z:\Drivers\wsname.exe /N:$DNS /NOREBOOT
    #else
    REM pause
    #end if
    echo Windows Registry Editor Version 5.00 > %systemdrive%\TMP\install.reg
    echo [HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce] >> %systemdrive%\TMP\install.reg
    echo "DD"="C:\\TMP\\install.cmd" >> %systemdrive%\TMP\install.reg
    $SNIPPET('my/win_install_drivers')
    
    #if $distro_name == 'Win2k3-Server_EN-x64'
    start /wait z:\Win2K3-Server_EN-x64\cmpnents\r2\setup2.exe /q /a /sr
    start /wait y:\Windows\Win2003\IE8-WindowsServer2003-x64-ENU.exe /passive /update-no /norestart
    if %virt% equ NO REG IMPORT y:\Windows\Win2003\vm.reg
    #end if
    REG IMPORT %systemdrive%\TMP\install.reg
    net use Y: /delete
    net use Z: /delete
    %systemdrive%\TMP\sleep.exe 10
    exit
    
    :flag005
    for /f "tokens=*" %%i in (%systemdrive%\TMP\stage.dat) do set stage=%%i
    echo 1 > %systemdrive%\TMP\stage.dat
    REG IMPORT %systemdrive%\TMP\install.reg
    if %stage% neq 0 goto flag010
    net use Y: /delete
    net use Z: /delete
    shutdown -r -f -t 5
    exit
    
    :flag010
    if %stage% gtr 1 goto flag020
    echo 2 > %systemdrive%\TMP\stage.dat
    
    $SNIPPET('my/winzip')
    $SNIPPET('my/winrar')
    $SNIPPET('my/win_install_chrome')
    $SNIPPET('my/win_install_ffox')
    $SNIPPET('my/win_install_adacr')
    #if $distro_name in ( 'WinXp_EN-i386', 'Win2k3-Server_EN-x64' )
    $SNIPPET('my/win_install_office_2007')
    #else if $distro_name in (  'Win7_EN-x64', 'Win8_EN-x64' )
    $SNIPPET('my/win_install_office_2010')
    
    < .. >
    
    Title Cleaning Temp files
    DEL "%systemroot%\*.bmp" >nul 2>&1
    DEL "%systemroot%\Web\Wallpaper\*.jpg" >nul 2>&1
    DEL "%systemroot%\system32\dllcache\*.scr" >nul 2>&1
    DEL "%systemroot%\system32\*.scr" >nul 2>&1
    DEL "%AllUsersProfile%\Start Menu\Windows Update.lnk" >nul 2>&1
    DEL "%AllUsersProfile%\Start Menu\Set Program Access and Defaults.lnk" >nul 2>&1
    DEL "%AllUsersProfile%\Start Menu\Windows Catalog.lnk" >nul 2>&1
    DEL "%systemdrive%\Microsoft Office*.txt" >nul 2>&1
    net user aspnet /delete >nul 2>&1
    REM %systemdrive%\TMP\sleep.exe 60
    net use Y: /delete
    net use Z: /delete
    
    shutdown -r -f -t 30
    RD /S /Q %systemdrive%\DRIVERS\ >nul 2>&1
    if not defined stage DEL /F /Q %systemdrive%\post_install.cmd
    DEL /F /S /Q %systemdrive%\TMP\*.*
    exit

If you need your own custom boot menu, add Windows to the network install menu in the /etc/cobbler/boot_loader_conf/pxedefault.template file:

  menu begin Windows
  MENU TITLE Windows
          label Win10_EN-x64
                  MENU INDENT 5
                  MENU LABEL Win10_EN-x64
                  kernel /winos/Win10_EN-x64/win10a.0
          label  Win10-profile1
                  MENU INDENT 5
                  MENU LABEL  Win10-profile1
                  kernel /winos/Win10_EN-x64/win10b.0
          label  Win10-profile2
                  MENU INDENT 5
                  MENU LABEL  Win10-profile2
                  kernel /winos/Win10_EN-x64/win10c.0
          label Win2016-Server_EN-x64
                  MENU INDENT 5
                  MENU LABEL Win2016-Server_EN-x64
                  kernel /winos/Win2016-Server_EN-x64/win6ra.0
  < .. >
          label returntomain
                  menu label Return to ^main menu.
                  menu exit
  menu end

Or create an iPXE boot menu

#!ipxe < .. > kernel http:///winos/wimboot initrd --name bootx64.efi http:///winos/Win10_EN-x64/EFI/Boot/bootx64.efi bootx64.efi initrd --name bcd http:///winos/Win10_EN-x64/boot/1Ea bcd initrd --name boot.sdi http:///winos/Win10_EN-x64/boot/boot.sdi boot.sdi initrd --name winpe.wim http:///winos/Win10_EN-x64/boot/winpe.wim winpe.wim boot < .. >

Final steps

Restart the services:

  systemctl restart cobblerd
  systemctl restart tftpd
  systemctl restart smb
  systemctl restart nmb

add distros:

  cobbler distro add –name=Win10_EN-x64 \
  --kernel=/var/lib/tftpboot/winos/Win10_EN-x64/pxeboot.n12 \
  --initrd=/var/lib/tftpboot/winos/Win10_EN-x64/boot/boot.sdi \
  --boot-loader=pxelinux \
  --arch=x86_64 --breed=windows –os-version=10 \
  --kernel-options='post_install=/var/lib/tftpboot/winos/Win10_EN-x64/sources/$OEM$/$1/post_install.cmd'

and profiles:

  cobbler profile add --name=Win10_EN-x64 --distro=Win10_EN-x64 --autoinstall=win.ks \
  --kernel-options='pxeboot=win10a.0, bootmgr=boot1ea.exe, bcd=1Ea,winpe=winpe.wim, sif=autounattended.xml'
    
  cobbler profile add --name=Win10-profile1 --parent=Win10_EN-x64 \
  --kernel-options='pxeboot=win10b.0, bootmgr=boot1eb.exe, bcd=1Eb,winpe=winp1.wim, sif=autounattende1.xml'
    
  cobbler profile add --name=Win10-profile2 --parent=Win10_EN-x64 \
  --kernel-options='pxeboot=win10c.0, bootmgr=boot1ec.exe, bcd=1Ec,winpe=winp2.wim, sif=autounattende2.xml'

cobbler sync
Install Windows