I have been struggling with this issue too! After some time debugging my kernel oops I was able to trace it back to this line of code:
$ gdb vmlinux
[snip]
(gdb) l *mod_sysfs_setup+0x36c
0xffff8000081152b0 is in mod_sysfs_setup (kernel/module/sysfs.c:263).
258 #ifdef CONFIG_MODULE_UNLOAD
259 struct module_use *use;
260
261 mutex_lock(&module_mutex);
262 list_for_each_entry(use, &mod->target_list, target_list) {
263 ret = sysfs_create_link(use->target->holders_dir,
264 &mod->mkobj.kobj, mod->name);
265 if (ret)
266 break;
267 }
(About `vmlinux`, apparently you should be able to decompress `vmlinuz` from the linux-image package to get this file, but I had no luck trying this. You can google it and give it a shot yourself if you like. I had been rebuilding my kernel to debug this issue so I was able to reuse the decompressed kernel image from `./cache/sources/linux-kernel-worktree/6.1__rk35xx__arm64/vmlinux` inside the armbian build dir)
It appears that the offending code is trying to create a symlink inside sysfs, and furthermore this behavior is controlled by the kernel config bit CONFIG_MODULE_UNLOAD. If you disable this bit, you will no longer be able to unload kernel modules while the system is running. I rebuilt the kernel with CONFIG_MODULE_UNLOAD unset and confirmed that inserting spl/zfs modules no longer triggers a kernel oops!
But this can hardly be considered anything more than a temporary workaround. There is some other root cause going on here. I hypothesize that something is trashing the kernel's module or sysfs state somewhere, but I haven't made much headway into discovering what this could be. I did find a couple of instances of modules in the downstream vendor kernel doing irresponsible things with sysfs, including a lurking double-free for which I will submit a PR in due time, but none of these appear to be the bug I'm looking for.
I should note that I'm not using zfs-dkms. I am building and installing ZFS from source from tag `zfs-2.3.0` instead. When I started debugging this, the arm64 package in ppa:arter97/zfs (which is mirrored by Armbian repositories) was partially missing, which prevented zfs-dkms and zfsutils-linux from being installed at the same time without downgrading. The arm64 build seems to be particularly flaky according to the build logs. This is to say that YMMV heavily, and I don't feel like re-testing this workaround with dkms yet, so if anyone else is interested enough to try this, let me know your results. FWIW the packages in the PPA appear to be fixed now.