rockboxdev.sh doesn't seem to run on Mac OS?

[Mudb0y]

When I try to run tools/rockboxdev.sh on my M2 MacBook Pro running Senoma, I get no output from my terminal. I ran it both as sudo and a normal user. I seem to have all the dependencies installed. Is there any recent Mac OS build issues I should be aware of?

[speachy]

to be honest, I don't expect rockoxdev.sh (or compiling rockbox even after a toolchain is present) to work on Macs; they have some pretty major differences in their environments from modern-ish Linux system and none of the active developers even have access to a mac, much less are able to test things out there.

But the fact you saw no output at all to be surprising, at minumum there should be _some_ output, even if only an error message.

Try running it with 'bash -x rockboxdev.sh' instead and see where it stops.

[chris_s]

You can replace readlink with greadlink and it should output something at least, but actually compiling the cross compiler will still fail on Apple Silicon at least.

[chris_s]

I would recommend spinning up either the latest ARM server versions of Debian or Ubuntu in VMWare Fusion (which is free for personal use) and you'll be up and running in no time. You can ssh into the VM or just use Fusion's own file sharing mechanism to transfer files back and forth.  Optionally install a desktop environment as well, if you intend to use the simulator on Linux.

https://blogs.vmware.com/teamfusion/2023/07/vmware-fusion-2023-tech-preview.html

[rockbox_dev123]

I made the greadlink changes myself and then gave up when I patched everything and realised as chris said, that the cross compilation fails. I went down the rabbit hole and tried different compilers but eventually got out of my depth.

There is a Docker image for compiling rockbox called which can be used ("built1n/rbclient").

However, this container is using a very old version of Debian and it actually ships the wrong version of the cross compilation binaries for arm targets anyway...

Here is my docker-compose setup for building rockbox:

Code: [Select]

  rockbox_build:
    image: debian
    container_name: rockbox_build
    entrypoint: /bin/bash
    command: "-c 'while true; do sleep 60; done'"
    volumes:
      - ${docker_config_path}/rockbox_build:/config
    restart: unless-stopped

This container will then happily idle forever until you run:

Code: [Select]

docker exec -it rockbox_build bash
Once inside the debian container, you can run this script to build rockbox for you:

Code: [Select]

#!/usr/bin/env bash

github_user_name="rockbox"
github_repo_name="rockbox"
#target="ipod6g"
arm_targets=("ipodvideo" "ipod6g")
rockbox_dir="/config/rockbox"
build_dir="${rockbox_dir}/build"

cd /config || exit

install_dependencies()
{
    apt update

    apt-get install -y \
        build-essential \
        git \
        perl \
        curl \
        texinfo \
        flex \
        bison \
        bzip2 \
        gzip \
        zip \
        patch \
        automake \
        libtool \
        libtool-bin \
        autoconf \
        libmpc-dev \
        gawk \
        python3 \
        python3-lzo \
        python3-setuptools \
        mtd-utils \
        xorriso \
        wget \
        subversion \
        libncurses5-dev \
        texlive-latex-base \
        texlive-binaries \
        texlive-latex-extra \
        tex4ht \
        texlive-fonts-recommended \
        lmodern \
        texlive-base \
        libsdl1.2-dev \
        libsdl1.2debian
        #latex-xcolor \
}

update_git_repo()
{
    if [[ ! -d "$rockbox_dir" ]]
    then
        git clone "https://github.com/${github_user_name}/${github_repo_name}.git"
    else
        cd "$rockbox_dir" || exit
        #git pull origin master
        git remote update
        if git status | grep -q 'Your branch is behind'
        then
            git pull origin master
            cd /config || exit
        fi
    fi
}

check_gcc_version()
{
    if ! command -v arm-elf-eabi-gcc >/dev/null || ! arm-elf-eabi-gcc -v 2>&1 | grep -q 'version 4.9.4'; then return 1; fi
}

initialise_toolchain()
{
    cd "$rockbox_dir" || exit
    #shellcheck disable=SC2016
    sed -i 's/tool="$tool -fLo "/tool="$tool -fLko "/' ./tools/rockboxdev.sh ## patch curl to allow invalid certs
    #shellcheck disable=SC2016
    sed -i 's/tool="$tool-T 60 -O "/tool="$tool --no-check-certificate -T 60 -O "/' ./tools/rockboxdev.sh ## patch wget to allow invalid certs

    mkdir -p "$rockbox_cross_compiler_dir"
    ./tools/rockboxdev.sh --target="a" "--prefix=$rockbox_cross_compiler_dir"
}

verify_toolchain()
{
    ## Check if the cross compilation binary exists, and if it is the right version.
    ## The rbclient docker image supplies the older version of 4.4.4 instead of 4.9.4
    ## 1) Check if the image has the right version (with default PATH)
    ## 2) Check if we have previously compiled the toolchain to a dir
    ## 3) Finally, compile the toolchain if necessary.
    if ! check_gcc_version
    then
        rockbox_cross_compiler_dir="/config/toolchain"
        export PATH="${rockbox_cross_compiler_dir}/bin:$PATH"
        if ! check_gcc_version
        then
            initialise_toolchain
        fi
    fi
}

build_rockbox()
{
    target="$1"
    mkdir -p "$build_dir" && cd "$build_dir" || exit

    ## Build rockbox
    ../tools/configure --target="$target" --type="n"
    make clean
    make -j"$(nproc)"

    ## Make the zip
    zip_name="rockbox-full.zip"
    make fullzip
    custom_zip_name="$(basename "$zip_name" .zip)-$target-$github_user_name-$(date +'%Y%m%d').zip"
    cp "$zip_name" "/config/$custom_zip_name"
}

build_bootloader()
{
    target="$1"
    mkdir -p "$build_dir" && cd "$build_dir" || exit

    ## Make the bootloader
    ../tools/configure --target="$target" --type="b"
    make clean
    make -j"$(nproc)"
    cp "bootloader-$target.ipod" "/config/bootloader-$target-$github_user_name-$(date +'%Y%m%d').ipod"
}

install_dependencies
update_git_repo
verify_toolchain

for build_target in "${arm_targets[@]}"
do
    build_rockbox "$build_target"
    build_bootloader "$build_target"
done


[speachy]

I made the greadlink changes myself and then gave up when I patched everything and realised as chris said, that the cross compilation fails. I went down the rabbit hole and tried different compilers but eventually got out of my depth.

I have a toolchain bump to GCC 8.5 sitting in gerrit, that should have a better shot of successfully building on MacOS.  (I've been holding it off until we get the 4.0 release out)

(See https://gerrit.rockbox.org/r/c/rockbox/+/4676 )

[Mudb0y]

Thanks for all the suggestions! I think I'll just go with the Linux VM, it seems like the easiest approach. Say, is there a way to compile a Mac OS simulator build from Linux? I know people have been building sims for recent Mac OS and I wonder how they did that if the toolchain doesn't run on the platform it self.

[speachy]

Simulator builds typically use the platform's native toolchain rather than a cross-compiler.  (Except Windows, because it's a special snowflake)

That said, there is a series of WIP patches in gerrit to greatly improve MacOS simulator situation. 

[rockbox_dev123]

I made the greadlink changes myself and then gave up when I patched everything and realised as chris said, that the cross compilation fails. I went down the rabbit hole and tried different compilers but eventually got out of my depth.

I have a toolchain bump to GCC 8.5 sitting in gerrit, that should have a better shot of successfully building on MacOS.  (I've been holding it off until we get the 4.0 release out)

(See https://gerrit.rockbox.org/r/c/rockbox/+/4676 )

Nice.

I did the following:

Code: [Select]

$ git clone  https://gerrit.rockbox.org/r/rockbox
$ cd rockbox/
$ git fetch https://gerrit.rockbox.org/r/rockbox refs/changes/76/4676/34 && git checkout -b change-4676 FETCH_HEAD
$ sed -i '' 's/readlink/greadlink/g' tools/rockboxdev.sh

I had to add "gmp" to the $needs_libs variable passed to the "build" function (for target "a") to get it to find the gmp include directory correctly.
e.g.

Code: [Select]

- build "binutils" "arm-elf-eabi" "2.37" "" "$binopts --disable-werror" "isl"
+ build "binutils" "arm-elf-eabi" "2.37" "" "$binopts --disable-werror" "gmp isl"

This allows me to build binutils.

However, the compilation of gcc fails.

I remember now that this was where I gave up when I last tried this.

The relevant gcc failure from the log file:

Code: [Select]

checking dependency style of g++... /bin/sh ../../gcc-8.5.0/gcc/../move-if-change tmp-mlib.h multilib.h
echo timestamp > s-mlib
Undefined symbols for architecture arm64:
  "_arm_abi", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch3m", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch4", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch5", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch5e", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch6", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch6k", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch6m", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch7", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch7em", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch8", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch_arm_hwdiv", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch_iwmmxt", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch_lpae", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch_notm", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_arch_thumb2", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_base_arch", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_disable_literal_pool", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_arm_restrict_it", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_flag_pic", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_operands", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_optimize_size", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_reload_completed", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_reload_in_progress", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_target_flags", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_this_target_rtl", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_unaligned_access", referenced from:
      ___cxx_global_var_init in gencondmd.o
  "_use_cmse", referenced from:
      ___cxx_global_var_init in gencondmd.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [Makefile:2870: build/gencondmd] Error 1

EDIT: I believe gcc would need to be updated to at least 11 to have arm64-apple-darwin support.

[speachy]

EDIT: I believe gcc would need to be updated to at least 11 to have arm64-apple-darwin support.

Well, that's disappointing.  We'll get there eventually though; I expect after GCC8 I'll shoot for 12.   Surprisingly, the hosted targets are the problematic ones as we have to stick to old libc to maintain ABI compatibility with the existing stuff.

[rockbox_dev123]

When I try to run tools/rockboxdev.sh on my M2 MacBook Pro running Senoma, I get no output from my terminal.

For a little more info, this is becase tools/rockboxdev.sh has "set -e" on line 7. This means that Bash will exit immediately if (to simplify) a command fails. The failure and exit happens before anything is printed.
See here for more info.

As well as making many commits to update the toolchain, speachy has good advice:

Try running it with 'bash -x rockboxdev.sh' instead and see where it stops.

Another way to do the same thing is to change the "set -e" line to "set -ex" and we see that in this case, chris_s is right that the calls to readlink are failing:

replace readlink with greadlink

I have successfully compiled rockbox for an ipodvideo on macOS Sequoia 15.1.1
I'm listening to music on the device now, so it has worked.

I have provided some steps below that can be followed.
I assume you are already using homebrew.
This has only been tested for the arm targets.

Steps to compile the gcc cross compiler with clang on macOS (from the latest master):

Code: [Select]

mkdir -p build && cd build
sed -i '' 's/readlink/greadlink/g' ../tools/rockboxdev.sh
sed -i '' 's/mpfr-3.1.6/mpfr-4.1.0/g' ../tools/rockboxdev.sh
sed -i '' 's/mpc-1.0.1/mpc-1.2.1/g' ../tools/rockboxdev.sh
sed -i '' "s/CXXFLAGS='-std=gnu++03' run_cmd/CXXFLAGS='-std=gnu++03 -fbracket-depth=512' run_cmd/" ../tools/rockboxdev.sh
sed -i '' 's/"isl"/"gmp isl"/g' ../tools/rockboxdev.sh
sed -i '' 's%-/tmp%-$TMPDIR%g' ../tools/rockboxdev.sh

You can now run ../tools/rockboxdev.sh as normal e.g.:

Code: [Select]

../tools/rockboxdev.sh --target="a"

Personally, I prefer to put the compiled binaries in a custom directory and add it to PATH manually e.g.

Code: [Select]

rockbox_cross_compiler_dir=$HOME/some/path
../tools/rockboxdev.sh --target="a" --prefix="$rockbox_cross_compiler_dir"
export PATH="${rockbox_cross_compiler_dir}/bin:$PATH"

This has the benefit of not requiring root to build (as you don't need to access the default prefix directory /usr/local).
I haven't tested running ../tools/rockboxdev.sh as root, but it probably works.

Once you get to the compilation stage for gcc, it will fail.
cd into the build directory (this will be $TMPDIR/rbdev-build/build-gcc/ by default if you didn't specify --builddir to override).
You will need to manually apply the patch here to $TMPDIR/rbdev-build/gcc-4.9.4/gcc/config/host-darwin.c.
You can skip the patch changes to $TMPDIR/rbdev-build/gcc-4.9.4/gcc/config/aarch64/aarch64.h.
After patching, run make. Compilation takes a phenomenally long time on my M1 Macbook Pro. So instead of making a coffee, consider cooking some dinner!
After compilation finishes, run (sudo) make install.

Finally, you can now compile rockbox (optionally specifying the prefix if you used a custom one):

Code: [Select]

target="ipodvideo"
../tools/configure --target="$target" --type="n" --prefix="$rockbox_cross_compiler_dir"
make fullzip

You should now have a build/rockbox-full.zip !

EDIT: gcc compilation time:

Code: [Select]

real 25m6.665s
user 74m47.146s
sys 32m44.817s


[chris_s]

That's awesome!

This has only been tested for the arm targets.

Can confirm this works for MIPS, too. An additional requirement in that case seems to be having gsed (GNU sed) installed and making sure it is used over the MacOS-provided one.

[speachy]

Code: [Select]

sed -i '' 's/mpfr-3.1.6/mpfr-4.1.0/g' ../tools/rockboxdev.sh
sed -i '' 's/mpc-1.0.1/mpc-1.2.1/g' ../tools/rockboxdev.sh


Major version bumps in these libraries could introduce codegen differences versus what's already in use, I don't want to bump those requirements for consistency's sake.
(They're already bumped as part of the GCC 9.5 toolchain upgrade, but there are still some issues there)

Meanwhile, I've applied all of the other changes in a way that preserves the ability to build on Linux.  Here's the patch; I'd appreciate someone making sure it actually works on MacOS too (after applying those two changes above manually)

https://gerrit.rockbox.org/r/c/rockbox/+/6134

EDIT:  I'm 98% sure that the "hosted" (ie X / Y ) tooclchains will not build properly under MacOS, due to their reliance on having system libraries present.

[rockbox_dev123]

Major version bumps in these libraries could introduce codegen differences versus what's already in use, I don't want to bump those requirements for consistency's sake.
(They're already bumped as part of the GCC 9.5 toolchain upgrade, but there are still some issues there)

Meanwhile, I've applied all of the other changes in a way that preserves the ability to build on Linux.  Here's the patch; I'd appreciate someone making sure it actually works on MacOS too (after applying those two changes above manually)

I downloaded an archive of your patch from Gerrit and replaced tools/rockboxdev.sh and added the tools/toolchain-patches/apple_silicon.patch file.

I also did these manual changes:

Code: [Select]

sed -i '' 's/mpfr-3.1.6/mpfr-4.1.0/g' ../tools/rockboxdev.sh
sed -i '' 's/mpc-1.0.1/mpc-1.2.1/g' ../tools/rockboxdev.sh


I can confirm that I am able to build the toolchain on macOS. I can't vote on the patch because I don't have the privileges. I've voted on the patch and would appreciate if this was merged into master.

Looking forward to the eventual GCC 9.5 toolchain upgrade. But until then I am happy to be able to build and I'm aware that I should probably switch to rockbox built on Linux (with the correct mpfr and mpc versions) before submitting any bug reports.

Thanks to all for your work.