Yocto Project : : Open Source

meta-rust

Jay.P Morgan 2024. 6. 1. 05:24

 

 

 

solana build

  • meta-rust
    • rust 관련 build를 위한 레이어
    • yocto 4.0 (kirkstone)에서, openembedded-core에 정식 합류
  • solana prerequisite (github)
    $ sudo apt-get install libssl-dev libudev-dev pkg-config zlib1g-dev llvm clang cmake make libprotobuf-dev protobuf-compiler

solana 관련 레시피 작성

  1. solana clone
$ git clone https://github.com/solana-labs/solana

  • target: solana/cli, solana/keygen
  • git uri: git://github.com/solana-labs/solana.git
  • commit: 05ffda9b16bfa9c784b22bbb0483f5a567373ae5

solana.inc + solana-cli_${PV}.bb

현재까지 문제

[RFC,1/2] cargo-update-recipe-crates.bbclass: add a class to generate SRC_URI crate lists from Cargo.lock

  • crate://의 문제https://patchwork.yoctoproject.org/project/oe-core/patch/20220930175520.2850089-1-alex@linutronix.de/
  • rust는 dependency를 tar, git 으로 가져오는 경향 보다는, cargo를 사용하여 dependencies로 fetch
  • yocto의 bitbake는 위의 것을 SRC_URI에 crate://에 명시해 가져오는데, 이는 cargo를 사용하지 않는다.
    • wget 을 사용한다.
    • 우선 이러한 '리스트'를 만듦으로서, 레시피가 업데이트 될 때, 리스트된 것들을 업데이트하는데 문제를 일으킨다.
  • cargo-update-recipe-crates.bbclass는 이러한 업데이트를 지원케 한다.
    • python3-bcrypt
    • python 스크립트는 tomllib library와 관련있으며, 이는 Python 3.11에 있음
      • 따라서, oe-core에서 python3.11-rc2로 먼저 업데이트해야한다.


1. 레시피의 ${S}를 가져와
2. Cargo.lock을 발견하면, (아래 예시는 solana의 Cargo.lock)

['package'] 를 가져와서 source, source 내부에 crates.io가 있으면
crate://crates.io/<name>/<version> 를 줄줄이 단다.

[RFC] python3: update 3.10.6 -> 3.11.0rc2

  • 위에서 설명한대로, cargo-update-recipe-crates.bbclass를 사용하기 위해,
    python 3.11.0rc2로 업그레이드한 컴포넌트를 넣는다.

patch: https://patchwork.yoctoproject.org/project/oe-core/patch/20220914120421.551923-1-alex@linutronix.de/

+) cargo-update-recipe-crates.bbclass

diff: https://git.openembedded.org/openembedded-core-contrib/commit/?h=jansa/master&id=73236b0fbcdb72c817666238f85dfeb9819cd2e2

  • use 4 spaces
  • avoid trailing space
  • add CARGO_LOC_SRC_DIR to allow searching outside ${S}
  • use ${BPN} in output file name
diff --git a/meta/classes-recipe/cargo-update-recipe-crates.bbclass b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
index f90938c734..c25fe8ef3e 100644
--- a/meta/classes-recipe/cargo-update-recipe-crates.bbclass
+++ b/meta/classes-recipe/cargo-update-recipe-crates.bbclass
@@ -16,26 +16,30 @@
 addtask do_update_crates after do_patch
 do_update_crates[depends] = "python3-native:do_populate_sysroot"
 
+# The directory where to search for Cargo.lock files relative to the root directory ${S}
+CARGO_LOCK_SRC_DIR ??= ""
+
 do_update_crates() {
     nativepython3 - <<EOF
 
 def get_crates(f):
     import tomllib
-    c_list = 'SRC_URI += " \\ \n'
+    c_list = '# from %s' % os.path.relpath(f, '${S}')
+    c_list += '\nSRC_URI += " \\\'
     crates = tomllib.load(open(f, 'rb'))
     for c in crates['package']:
         if 'source' in c and 'crates.io' in c['source']:
-            c_list += "        crate://crates.io/{}/{} \\ \n".format(c['name'], c['version'])
-    c_list += '"\n'
+            c_list += '\n    crate://crates.io/%s/%s \\\' % (c['name'], c['version'])
+    c_list += '\n"\n'
     return c_list
 
 import os
 crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
-for root, dirs, files in os.walk('${S}'):
+for root, dirs, files in os.walk('${S}/${CARGO_LOCK_SRC_DIR}'):
     for file in files:
         if file == 'Cargo.lock':
             crates += get_crates(os.path.join(root, file))
-open(os.path.join('${THISDIR}', '${PN}'+"-crates.inc"), 'w').write(crates)
+open(os.path.join('${THISDIR}', '${BPN}'+"-crates.inc"), 'w').write(crates)
 
 EOF
 }

  • CARGO_LOCK_SRC_DIR

solana recipes

  • recipes-A/solana/

solana-program-library recipes

  • recipes-A/solana-program-library/
  • Error
DEBUG: Executing shell function do_compile
NOTE: cargo build -v --target aarch64-unknown-linux --release --manifest-path=TOPDIR/BUILD/work/raspberrypi4_64-unknown-linux/solana-program-library/2.1.1-r0/git/token/cli//Cargo.toml 
error: failed to load manifest for workspace member `TOPDIR/BUILD/work/raspberrypi4_64-webos-linux/solana-program-library/2.1.1-r0/git/token-upgrade/cli`

Caused by:
  failed to load manifest for dependency `spl-token-client`

Caused by:
  failed to parse manifest at `TOPDIR/BUILD/work/raspberrypi4_64-unknown-linux/solana-program-library/2.1.1-r0/git/token/client/Cargo.toml`

Caused by:
  namespaced features with the `dep:` prefix are only allowed on the nightly channel and requires the `-Z namespaced-features` flag on the command-line
WARNING: exit code 101 from a shell command.
  • Older cargo isn't compatible!
  • 또한, python3_pyo3.bbclass를 backport 해야한다.
    • build 과정 중 python3-cryptography에서 Error발생
      • error: Error loading target specification: Could not find specification for target "aarch64-unknown-linux". Run rustc --print target-list for a list of built-in targets
    • PyO3: https://github.com/PyO3/pyo3
      • rust에서 python을 사용 혹은 그 반대

rust-target-config

  • yocto 4.1의 rust-target-config.bbclass
    BUILD HOST(native)를 x86-64로 사용
    TUNE_FEATURES는 항상 target을 참조한다. 따라서 BUILD HOST(native)에서는 사용하길 원치않는다.
    TUNE_FEATURES에 vfpv4가 있으면, f에 +vfp4를 추가한다.
    ....

rust-common.bbclass

  • yocto 4.1의 rust-common.bbclass
    rustlibdir = "${libdir}/rustlib/${RUST_HOST_SYS}/lib
    • FILES:${PN} += "${rustlibdir}/*.so"
    • FILES:${PN}-dev += "${rustlibdir}/*.rlib ${rustlibdir}/*.rmeta "
    • FILES:${PN}-dbg += "${rustlibdir}/.debug"
  • RUSTLIB = "-L ${STAGING_DIR_HOST}${rustlibdir}"
  • RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
  • RUSTFALGS += "${RUSTLIB} ${RUST_DEBUG_REMAP}"
  • RUSTLIB_DEP ?= "libstd-rs"
  • RUST_PANIC_STRATEGY ?= "unwind"

  • RUST_LIBC는 제거 되었음, backport된 레시피는 필요치 않음. 하지만 overlayed bbclass는 kirkstone의 원래 rust-crosssdk 레시피 parsing에 실패하게 만든다.
    • 이 레이어의 1.63.0 버전이 PREFERRED 될 것이기 때문에 결국엔 사용되지는 않을 것임
    • rust-crosssdk_1.59.0.bb
  • rust_base_triple
    • Mangle bitbake's *_SYS into something that rust might support (see rust/mk/cfg/* for a list
    • Note that os is assumed to be some linux form
  • RUST_BUILD_ARCH = "x86_64"
  • Naming explanation
    • Yocto
      • BUILD_SYS: Yocto triple of the build environment
      • HOST_SYS: What we're building for in Yocto
      • TARGET_SYS: What we're building for in Yocto
      • So when building -native
        • BUILD_SYS == HOST_SYS == TARGET_SYS
      • 일반적인 Component
        • HOST_SYS == TARGET_SYS
    • Rust
      • TARGET: the system where the binary will run
      • HOST : the system whrere the binary is being built

build-rust-cc

#!/usr/bin/env python3
import os, sys
orig_binary = "gcc "
extras = ""
binary = orig_binary.split()[0]
args = orig_binary.split() + sys.argv[1:]
if extras:
	args.append(extras)
os.execvp(binary, args)

target-rust-cc

#!/usr/bin/env python3
import os, sys
orig_binary = "aarch64-webos-linux-gcc  -mcpu=cortex-a72 -march=armv8-a+crc -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security -Werror=return-type --sysroot=/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-webos-linux/solana-cli/1.14.5-r0/recipe-sysroot -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fmacro-prefix-map=/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-webos-linux/solana-cli/1.14.5-r0=/usr/src/debug/solana-cli/1.14.5-r0                      -fdebug-prefix-map=/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-webos-linux/solana-cli/1.14.5-r0=/usr/src/debug/solana-cli/1.14.5-r0                      -fdebug-prefix-map=/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-webos-linux/solana-cli/1.14.5-r0/recipe-sysroot=                      -fdebug-prefix-map=/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-webos-linux/solana-cli/1.14.5-r0/recipe-sysroot-native=  -Wl,-z,relro,-z,now"
extras = ""
binary = orig_binary.split()[0]
args = orig_binary.split() + sys.argv[1:]
if extras:
    args.append(extras)
os.execvp(binary, args)

solana workdir

${CARGO_HOME}/config

# EXTRA_OECARGO_PATHS
paths = [
]

# Local mirror vendored by bitbake
[source.bitbake]
directory = "/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-<os>-linux/solana-cli/1.14.5-r0/cargo_home/bitbake"

[source.crates-io]
replace-with = "bitbake"
local-registry = "/nonexistant"

[http]
# Multiplexing can't be enabled because http2 can't be enabled
# in curl-native without dependency loops
multiplexing = false

# Ignore the hard coded and incorrect path to certificates
cainfo = "/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-<os>-linux/solana-cli/1.14.5-r0/recipe-sysroot-native/etc/ssl/certs/ca-certificates.crt"


# HOST_SYS
[target.aarch64-webos-linux-gnu]
linker = "/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-<os>-linux/solana-cli/1.14.5-r0/wrapper/target-rust-ccld"

# BUILD_SYS
[target.x86_64-linux-gnu]
linker = "/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-<os>-linux/solana-cli/1.14.5-r0/wrapper/build-rust-ccld"

[build]
# Use out of tree build destination to avoid poluting the source tree
target-dir = "/home/dhyang/workspace/build-webos/BUILD/work/raspberrypi4_64-<os>-linux/solana-cli/1.14.5-r0/build/target"

[term]
progress.when = 'always'
progress.width = 80

openssl-sys 문제

thread 'main' panicked at 'don't know how to configure OpenSSL for \
aarch64-webos-linux-gnu', \
TOPDIR/BUILD/work/raspberrypi4_64-<os>-linux/solana-program-library/2.1.1-r0/\
cargo_home/bitbake/openssl-src-111.22.0+1.1.1q/src/lib.rs:294:18
  stack backtrace:
     0: rust_begin_unwind
  • solana-program-library에 DEPENDS에 openssl을 추가하고, export OPENSSL_NO_VENDOR = "1" 문장 추가.

 

 

 

출처: https://velog.io/@markyang92/posts?tag=Yocto

'Yocto Project : : Open Source' 카테고리의 다른 글

ADVANCED: do_package  (0) 2024.06.01
meta-rust-bin, fakeroot  (0) 2024.06.01
clang-sys  (0) 2024.06.01
LICENSE, LIC_FILES_CHKSUM, COMMON_LICENSE_DIR, LICENSE_FLAGS_ACCEPTED  (0) 2024.06.01
core-image-sato, SDK, TOOLCHAIN_HOST_TASK, populate_sdk, gdb, gdbserver, dev 패키지, debug tools 설치, PACKAGE_DEBUG_SPLIT_STYLE  (0) 2024.06.01

'Yocto Project : : Open Source'의 다른글

  • 현재글meta-rust

관련글

티스토리툴바