libc/builtin: support per-application priority/stacksize under KERNEL…

[hitHuang]

Summary

I'm porting NuttX to a SiFive U74 core that currently runs Linux; we're moving to an RTOS
there mainly to cut memory footprint. For reasons specific to this port we're using
CONFIG_BUILD_KERNEL, and while adapting existing applications to it I found that an
application's Kconfig-configured priority and stack size (e.g.
CONFIG_EXAMPLES_HELLO_PRIORITY / CONFIG_EXAMPLES_HELLO_STACKSIZE) had no effect at all
under CONFIG_BUILD_KERNEL — every app was spawned with the same default priority/stack
size regardless of what was configured.

The root cause is that nsh_fileapp() looks up an app's configured priority/stacksize in
the compile-time registry table (struct builtin_s / g_builtins[]) before calling
posix_spawn(), but that table was only built under CONFIG_BUILTIN, and
CONFIG_BUILTIN depends on !BUILD_KERNEL. So under CONFIG_BUILD_KERNEL the table simply
didn't exist, and the lookup silently never ran.

A companion PR on nuttx-apps (nshlib: apply configured priority/stacksize under CONFIG_BUILD_KERNEL) switches apps/builtin/Make.defs and apps/nshlib/nsh_fileapps.c
to the same CONFIG_APP_REGISTRY symbol so nsh_fileapp() can actually perform the lookup
under CONFIG_BUILD_KERNEL. This nuttx-side PR is self-contained and safe to merge on
its own — APP_REGISTRY is purely additive and defaults to the exact same behavior as
today's CONFIG_BUILTIN for every existing FLAT/PROTECTED configuration; the apps-side PR
should be submitted only after this one merges, since it changes an ifneq guard that
currently resolves via CONFIG_BUILTIN and would otherwise regress against nuttx master
(which doesn't have CONFIG_APP_REGISTRY yet).

Separately, I'd like to ask the community directly: is CONFIG_BUILD_KERNEL considered a
second-class citizen at this point? Most of the gaps I've been running into while porting
(this one included) come from code paths that quietly assume FLAT or PROTECTED and were
never exercised under KERNEL build. If the general recommendation is to prefer FLAT/
PROTECTED unless strict process isolation is a hard requirement, that would be useful to
know up front — but if CONFIG_BUILD_KERNEL is meant to be fully supported, I'd like to
keep sending fixes like this one as I find them.

Impact

• New Kconfig symbol APP_REGISTRY, hidden (no prompt), purely derived. No effect on any
  existing configuration: it evaluates to the same condition CONFIG_BUILTIN already
  satisfied under FLAT/PROTECTED, plus CONFIG_BUILD_KERNEL.
• Under CONFIG_BUILD_KERNEL with CONFIG_SCHED_USER_IDENTITY=y, the uid/gid/mode
  fields of struct builtin_s are now also populated (previously the whole struct didn't
  exist in that build mode). They remain unused: their only consumer,
  fs/binfs/fs_binfs.c, is reached only through FS_BINFS depends on BUILTIN, which is
  still unreachable under CONFIG_BUILD_KERNEL. This is inert, not a behavior change.
• No change to CONFIG_BUILTIN, its dependency on !BUILD_KERNEL, or any of the
  main_t-dispatch call sites.
• Requires the companion nuttx-apps PR to actually take effect; without it, the new table
  is built but nothing reads it yet (same as today).

Testing

Host: Linux

Target: QEMU RISC-V (rv-virt), config knsh_romfs, with the default
CONFIG_EXAMPLES_HELLO_STACKSIZE=8192 removed so the app's own Kconfig defaults apply.

Test app: modified examples/hello:

int main(int argc, FAR char *argv[])
{
  printf("Hello, World!!\n");
  sleep(3);
  return 0;
}

with apps/examples/hello/Kconfig changed to non-default values to make any effect
observable:

 config EXAMPLES_HELLO_PRIORITY
        int "Hello task priority"
-       default 100
+       default 120

 config EXAMPLES_HELLO_STACKSIZE
        int "Hello stack size"
-       default DEFAULT_TASK_STACKSIZE
+       default 1000

Before this fix (plus the companion apps PR), hello's configured priority (120) and
stack size (1000) are ignored; it runs with priority 100 and a much larger stack:

NuttShell (NSH) NuttX-13.0.0-RC2
nsh> hello &
hello [0:100]
nsh> Hello, World!!
ps
  TID   PID  PPID PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK            STACK COMMAND
    0     0     0   0 FIFO     Kthread   - Ready              0000000000000000 0003040 Idle_Task
    1     0     0 100 RR       Kthread   - Waiting  Semaphore 0000000000000000 0001968 lpwork 0x80600010 0x80600060
    3     3     0 100 RR       Task      - Running            0000000000000000 0003008 /system/bin/init
    4     4     3 100 RR       Task      - Waiting  Signal    0000000000000000 0008144 hello

After this fix (with the companion apps PR applied), the configured priority (120) and
stack size (1000, rounded to 944 usable after bookkeeping) are correctly applied:

NuttShell (NSH) NuttX-13.0.0-RC2
nsh> hello &
Hello, World!!
hello [0:100]
nsh> ps
  TID   PID  PPID PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK            STACK COMMAND
    0     0     0   0 FIFO     Kthread   - Ready              0000000000000000 0003040 Idle_Task
    1     0     0 100 RR       Kthread   - Waiting  Semaphore 0000000000000000 0001968 lpwork 0x80600010 0x80600060
    3     3     0 100 RR       Task      - Running            0000000000000000 0003008 /system/bin/init
    4     4     3 120 RR       Task      - Waiting  Signal    0000000000000000 0000944 hello

Also re-verified with rv-virt:nsh (CONFIG_BUILTIN=y, CONFIG_BUILD_KERNEL not set) that
existing FLAT-mode CONFIG_APP_REGISTRY behavior (derived from CONFIG_BUILTIN) is
unchanged — builtin lookup and priority/stacksize handling behave exactly as before this
change.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• arduino-mega2560
• esp32-devkitc
• hifive1-revb
• mirtoo
• qemu-armv8a
• qemu-intel64
• rx65n-rsk2mb
• s698pm-dkit
• stm32-nucleo-f103rb

[xiaoxiang781216]

need update cmake too

[xiaoxiang781216]

the more natural approach is attached the config as elf symbol, so elf binfmt could initialize them during loading.

[linguini1]

but if CONFIG_BUILD_KERNEL is meant to be fully supported, I'd like to keep sending fixes like this one as I find them.

I think the reason you see most of these issues is because the majority of the devices NuttX is being used on are running the flat build. However, the kernel build is meant to be fully supported and these issues need to fixed. Especially as we're starting to see more targets that have hardware to support kernel builds. So good find!