Using b43-fwcutter on Windows drivers

[João Paulo Rechi Vita]

Hello b43 developers,

Is it possible to use b43-fwcutter to extract firmware from Windows
drivers? The manpage says "Currently b43-fwcutter supports Apple MacOS
X, Microsoft Windows and Linux drivers, but keep in mind that
b43-fwcutter doesn't support all driver versions.", but looking at
fwcutter_list.h I don't see entries for the windows drivers. Are
offsets for windows drivers available somewhere?

Thanks,

......................................................................................
João Paulo Rechi Vita | +1.415.851.5778 | Endless

[Larry Finger]

Post by João Paulo Rechi Vita
Hello b43 developers,
Is it possible to use b43-fwcutter to extract firmware from Windows
drivers? The manpage says "Currently b43-fwcutter supports Apple MacOS
X, Microsoft Windows and Linux drivers, but keep in mind that
b43-fwcutter doesn't support all driver versions.", but looking at
fwcutter_list.h I don't see entries for the windows drivers. Are
offsets for windows drivers available somewhere?

The reason that you do not see any entries for Windows drivers is because those
drivers are stripped such that we can get the offsets, but not the description
for that particular firmware entry. One could go through each one and identify
the correspondence with other firmware sources, but that would be very laborious
and error prone. I have chosen not to try that approach.

What do you hope to gain from Windows firmware? Extracting firmware from the
Linux drivers should yield better firmware if the two operating systems have
different requirements.

Larry

[João Paulo Rechi Vita]

Post by Larry Finger

Post by João Paulo Rechi Vita
Hello b43 developers,
Is it possible to use b43-fwcutter to extract firmware from Windows
drivers? The manpage says "Currently b43-fwcutter supports Apple MacOS
X, Microsoft Windows and Linux drivers, but keep in mind that
b43-fwcutter doesn't support all driver versions.", but looking at
fwcutter_list.h I don't see entries for the windows drivers. Are
offsets for windows drivers available somewhere?

The reason that you do not see any entries for Windows drivers is because
those drivers are stripped such that we can get the offsets, but not the
description for that particular firmware entry. One could go through each
one and identify the correspondence with other firmware sources, but that
would be very laborious and error prone. I have chosen not to try that
approach.
What do you hope to gain from Windows firmware? Extracting firmware from the
Linux drivers should yield better firmware if the two operating systems have
different requirements.

Thanks for your reply, Larry.

The idea is to help non-techy users on dual-boot installations, when
they have no other means to connect their Linux system to the
internet. On Endless we support a very specific Windows+Endless
dual-boot layout, so it is easy for us to find the Windows partition.
We want grab the Windows drivers and extract the firmware files from
them automatically when a compatible card is detected and the user is
offline.

I understand that there is some manual work involved on this, but are
there any pointers on how to get the offsets from the Windows drivers?
I could check if the extracted data matches anything we already know
from the wl firmware.

Also, you mention that firmware may be different due to different OS
requirements -- I expected the hw (thus, also the firmware) to behave
the same way despite which OS is running on the CPU. Maybe I'm being
naive here?

......................................................................................
João Paulo Rechi Vita | +1.415.851.5778 | Endless

[Larry Finger]

Post by João Paulo Rechi Vita

Post by Larry Finger

Post by João Paulo Rechi Vita
Hello b43 developers,
Is it possible to use b43-fwcutter to extract firmware from Windows
drivers? The manpage says "Currently b43-fwcutter supports Apple MacOS
X, Microsoft Windows and Linux drivers, but keep in mind that
b43-fwcutter doesn't support all driver versions.", but looking at
fwcutter_list.h I don't see entries for the windows drivers. Are
offsets for windows drivers available somewhere?

The reason that you do not see any entries for Windows drivers is because
those drivers are stripped such that we can get the offsets, but not the
description for that particular firmware entry. One could go through each
one and identify the correspondence with other firmware sources, but that
would be very laborious and error prone. I have chosen not to try that
approach.
What do you hope to gain from Windows firmware? Extracting firmware from the
Linux drivers should yield better firmware if the two operating systems have
different requirements.

Thanks for your reply, Larry.
The idea is to help non-techy users on dual-boot installations, when
they have no other means to connect their Linux system to the
internet. On Endless we support a very specific Windows+Endless
dual-boot layout, so it is easy for us to find the Windows partition.
We want grab the Windows drivers and extract the firmware files from
them automatically when a compatible card is detected and the user is
offline.
I understand that there is some manual work involved on this, but are
there any pointers on how to get the offsets from the Windows drivers?
I could check if the extracted data matches anything we already know
from the wl firmware.
Also, you mention that firmware may be different due to different OS
requirements -- I expected the hw (thus, also the firmware) to behave
the same way despite which OS is running on the CPU. Maybe I'm being
naive here?

In general, the firmware should be the same, but as I have not seen any
specifics on Windows drivers, there could be differences. The reverse
engineering started with MIPS drivers for Linux. We never looked at any RE for
Windows drivers.

A potential solution to your problem might be to use The method employed in
http://www.lwfinger.com/b43-firmware/no_net_install_bcm43xx_firmware.tar.bz2.
That tar file contains Broadcom object files with firmware for b43 and b43legacy
with a script that extracts the firmware files. It meets Broadcom restrictions
in that it only redistributes drivers, not firmware files. That particular tar
file contains 5.100.138 b43 firmware; however, it would be easy to package a
different driver, and adjust the extraction script accordingly. If it is not
possible to package the tar file as part of your Windows installation, at least
you could have your users download that file from a Windows boot, and then run
the firmware extraction from a Linux boot.

The only thing I ask is that you host the download on your server. I do not
promise to host those files in perpetuity. I wrote part of the script file, and
you have my permission to use it as you please as long as you preserve my
copyright. The rest of the script is part of the openSUSE package.

Larry

[João Paulo Rechi Vita]

On 06/07/2017 07:08 PM, João Paulo Rechi Vita
A potential solution to your problem might be to use The method employed in
http://www.lwfinger.com/b43-firmware/no_net_install_bcm43xx_firmware.tar.bz2.
That tar file contains Broadcom object files with firmware for b43 and
b43legacy with a script that extracts the firmware files. It meets Broadcom
restrictions in that it only redistributes drivers, not firmware files. That
particular tar file contains 5.100.138 b43 firmware; however, it would be
easy to package a different driver, and adjust the extraction script
accordingly. If it is not possible to package the tar file as part of your
Windows installation, at least you could have your users download that file
from a Windows boot, and then run the firmware extraction from a Linux boot.
The only thing I ask is that you host the download on your server. I do not
promise to host those files in perpetuity. I wrote part of the script file,
and you have my permission to use it as you please as long as you preserve
my copyright. The rest of the script is part of the openSUSE package.

We already ship a similar script based on the debian package for
b43-fwcutter (with some adaptations for our rootfs layout), that would
download the drivers, extract and install firmware for the user, so we
could just ship the files as well. What I was trying to avoid is
shipping those files, and that's where extracting from the Windows
drivers would come in handy.

Thanks,

......................................................................................
João Paulo Rechi Vita | +1.415.851.5778 | Endless

[Michael Büsch]

On Mon, 12 Jun 2017 12:28:09 -0400

Post by João Paulo Rechi Vita
We already ship a similar script based on the debian package for
b43-fwcutter (with some adaptations for our rootfs layout), that would
download the drivers, extract and install firmware for the user, so we
could just ship the files as well. What I was trying to avoid is
shipping those files, and that's where extracting from the Windows
drivers would come in handy.

Well, the problem is that it is very hard to get the names for the
firmware images found in the binary.

What you could do is try to find a Linux driver and a Windows driver
with exactly matching FW versions and then use the names from the Linux
driver to extract the blobs from the Windows driver.
It would be pretty easy to write a script that takes a Linux driver and
its matching extract-offsets and extract-names and then create the
corresponding offsets for the Windows binary.
Please feel free to create such a script and I would be happy to
include it in b43-fwcutter.

A script to auto-generate the extract-info from a Linux driver is
already included with b43-fwcutter.

[João Paulo Rechi Vita]

Hello again,

Post by Michael Büsch
On Mon, 12 Jun 2017 12:28:09 -0400

Post by João Paulo Rechi Vita
We already ship a similar script based on the debian package for
b43-fwcutter (with some adaptations for our rootfs layout), that would
download the drivers, extract and install firmware for the user, so we
could just ship the files as well. What I was trying to avoid is
shipping those files, and that's where extracting from the Windows
drivers would come in handy.

Well, the problem is that it is very hard to get the names for the
firmware images found in the binary.
What you could do is try to find a Linux driver and a Windows driver
with exactly matching FW versions and then use the names from the Linux
driver to extract the blobs from the Windows driver.

We've been trying to do some binary grepping for the first 4 bytes of
the firmware files in the windows driver (grep -obUaP
"\x75\x01\x00\x00" BCMWL664.SYS), but although there are some matching
occurrences, after that the bytes start to differ. Any other
suggestions on trying to match the firmware files with windows
drivers?

Post by Michael Büsch
It would be pretty easy to write a script that takes a Linux driver and
its matching extract-offsets and extract-names and then create the
corresponding offsets for the Windows binary.
Please feel free to create such a script and I would be happy to
include it in b43-fwcutter.
A script to auto-generate the extract-info from a Linux driver is
already included with b43-fwcutter.

Yep, I have looked at that, and tried to objdump the windows driver,
but as you guys said, the symbol names are stripped out of it.

On a another topic, is there a license file available somewhere for
the wl MIPS drivers? I've downloaded the Asus/Netgear tarballs that
they were originally extracted from, but couldn't find a license for
the drivers there, just a general release notes file on the top dir,
and some un-related licenses for different projects used in the
router's rootfs image.

Thanks,

......................................................................................
João Paulo Rechi Vita | +1.415.851.5778 | Endless

[Michael Büsch]

On Thu, 15 Jun 2017 18:21:40 -0400

Post by João Paulo Rechi Vita
We've been trying to do some binary grepping for the first 4 bytes of
the firmware files in the windows driver (grep -obUaP
"\x75\x01\x00\x00" BCMWL664.SYS), but although there are some matching
occurrences, after that the bytes start to differ. Any other
suggestions on trying to match the firmware files with windows
drivers?

Well, then you most likely have differing fw versions.
Try to find other binaries.

Post by João Paulo Rechi Vita
On a another topic, is there a license file available somewhere for
the wl MIPS drivers? I've downloaded the Asus/Netgear tarballs that
they were originally extracted from, but couldn't find a license for
the drivers there, just a general release notes file on the top dir,
and some un-related licenses for different projects used in the
router's rootfs image.

I don't know anything about the licensing.

[Robert McQueen]

Hi Larry,

Post by Larry Finger
A potential solution to your problem might be to use The method employed in
http://www.lwfinger.com/b43-firmware/no_net_install_bcm43xx_firmware.tar.bz2.
That tar file contains Broadcom object files with firmware for b43 and
b43legacy with a script that extracts the firmware files. It meets
Broadcom restrictions in that it only redistributes drivers, not
firmware files. That particular tar file contains 5.100.138 b43
firmware; however, it would be easy to package a different driver, and
adjust the extraction script accordingly. If it is not possible to
package the tar file as part of your Windows installation, at least you
could have your users download that file from a Windows boot, and then
run the firmware extraction from a Linux boot.

Thanks for this, I work with Joao and I've been investigating this
avenue of automatically fetching the drivers and then unpacking the
firmware when needed with a modprobe install hook, so we can get these
b43 cards working as seamlessly as possible in Endless when udev/etc
finds a supported card. I don't have any relevant hardware to test
unfortunately, but at least "on paper" it looks like it's working
nicely. Before we press on and enable it in the distro, I had a few
questions.

1. It looks like brcmsmac supports three devices which are all also
claimed by b43. I imagine that, because b43 usually requires manual
interaction with fwcutter, it's rare that users would get b43 working
unless they knew they needed it - so in a sense brcmsmac is the
"default" driver for these devices. However if we get b43 its firmware
automatically, does it become non-deterministic which driver will bind?

If so it feels to me (although I've got no evidence) that this could
expose some differences in behaviour / capabilities and could confuse
userland stuff like NetworkManager which uses the wireless extensions
extensively, so settings might not be applied correctly, etc. My
instinct is to just make this choice one way or another by either having
b43 not register for these three devices in our kernel, or blacklist
brcmsmac. If both drivers are available, do you have any suggestions
over which one would/should be used?

2. Your script uses the 5.100.138 driver, but 6.30.163.46 is available.
Is there any reason to not use the newer one? I had naively assumed
newer is better but there is some anecdotal evidence that some versions
work better for some cards - Arch Linux goes to the effort of packaging
two different firmware versions:
https://wiki.archlinux.org/index.php/broadcom_wireless#b43

Is there any truth/merit in this? Which firmware would you use in an
optimal configuration? Do I need to switch the version based on the
device, or get the firmware for some cards from an older version and
some from a newer one?

3. I've not been able to find the MIPS wl_apsta drivers for download
directly from Broadcom, and every other source (wifi router source
drops, OpenWRT, etc) I've found seems not to have a LICENSE file along
with the .o. It seems most likely to me that the same Broadcom license
as applied to the PC STA wl drivers applies to these ones - ie like
http://metadata.ftp-master.debian.org/changelogs/non-free/b/broadcom-sta/broadcom-sta_6.30.223.271-7_copyright
- but I have no proof of that. Does anyone have any pointers to a more
definitive statement of the license the wl_apsta.o drivers are under?

Post by Larry Finger
Larry

Cheers,
Rob

........................................................................

Robert McQueen | +1.415.413.4159 | Endless <http://endlessm.com/>

[Larry Finger]

Post by Robert McQueen
Thanks for this, I work with Joao and I've been investigating this
avenue of automatically fetching the drivers and then unpacking the
firmware when needed with a modprobe install hook, so we can get these
b43 cards working as seamlessly as possible in Endless when udev/etc
finds a supported card. I don't have any relevant hardware to test
unfortunately, but at least "on paper" it looks like it's working
nicely. Before we press on and enable it in the distro, I had a few
questions.
1. It looks like brcmsmac supports three devices which are all also
claimed by b43. I imagine that, because b43 usually requires manual
interaction with fwcutter, it's rare that users would get b43 working
unless they knew they needed it - so in a sense brcmsmac is the
"default" driver for these devices. However if we get b43 its firmware
automatically, does it become non-deterministic which driver will bind?

I would expect that the connection would not be deterministic for those devices.
The safest thing would be for you to create a special kernel patch that would
comment out the appropriate lines in the PCI device table found in
drivers/bcma/host_pci.c. That would be the entries with device ID of 0x0576,
0x4353, 0x4357, and 43224, although that last one does not seem to be in my
table. With those changes, there would be no contention.

Post by Robert McQueen
If so it feels to me (although I've got no evidence) that this could
expose some differences in behaviour / capabilities and could confuse
userland stuff like NetworkManager which uses the wireless extensions
extensively, so settings might not be applied correctly, etc. My
instinct is to just make this choice one way or another by either having
b43 not register for these three devices in our kernel, or blacklist
brcmsmac. If both drivers are available, do you have any suggestions
over which one would/should be used?

I suspect that the Broadcom version would be better. They have access to the
data sheets for the device, which the b43 authors do not.

Post by Robert McQueen
2. Your script uses the 5.100.138 driver, but 6.30.163.46 is available.
Is there any reason to not use the newer one? I had naively assumed
newer is better but there is some anecdotal evidence that some versions
work better for some cards - Arch Linux goes to the effort of packaging
https://wiki.archlinux.org/index.php/broadcom_wireless#b43
Is there any truth/merit in this? Which firmware would you use in an
optimal configuration? Do I need to switch the version based on the
device, or get the firmware for some cards from an older version and
some from a newer one?

The newer version may be better. We have no documentation regarding what changes
may have been made. You get to make your choice.

Post by Robert McQueen
3. I've not been able to find the MIPS wl_apsta drivers for download
directly from Broadcom, and every other source (wifi router source
drops, OpenWRT, etc) I've found seems not to have a LICENSE file along
with the .o. It seems most likely to me that the same Broadcom license
as applied to the PC STA wl drivers applies to these ones - ie like
http://metadata.ftp-master.debian.org/changelogs/non-free/b/broadcom-sta/broadcom-sta_6.30.223.271-7_copyright
- but I have no proof of that. Does anyone have any pointers to a more
definitive statement of the license the wl_apsta.o drivers are under?

You will only find those wl_apsta drivers contained within the open-source
packages for the various AP systems. That is where you would find any license
statements made by Broadcom.

Larry