查看问题详情

编号项目分类查看权限最后更新
0000322Anolis OS 7Generalpublic2021-11-17 10:41
报告员wb-wpp899309 分派给XuanZhuo  
优先级low严重性minor出现频率always
状态 assigned处理状况open 
平台x86_64操作系统Anolis OS操作系统版本7.7
标题0000322: [Anolis OS 7.7] 开机启动后,dmesg日志中有err日志
描述[缺陷描述]:
开机启动后,dmesg日志中有如下err日志:

# dmesg -l err -T
[Wed Oct 13 23:18:27 2021] integrity: Unable to open file: /etc/keys/x509_ima.der (-2)
[Wed Oct 13 23:18:27 2021] integrity: Unable to open file: /etc/keys/x509_evm.der (-2)
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:34 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:36 2021] bpfilter: read fail -13
[Wed Oct 13 23:18:37 2021] bpfilter: read fail -13


[重现概率]
必现

[重现环境]
# cat /etc/os-release
NAME="Anolis OS"
VERSION="7.7"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="7.7"
PRETTY_NAME="Anolis OS 7.7"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"
BUG_REPORT_URL="https://bugs.openanolis.cn/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

内核:
# uname -r
4.19.91-24.8.an7.x86_64

cpu信息:
# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: HygonGenuine
CPU family: 24
Model: 1
Model name: Hygon C86 7280 32-core Processor
Stepping: 1
CPU MHz: 1999.999
BogoMIPS: 3999.99
Virtualization: AMD-V
Hypervisor vendor: KVM
Virtualization type: full
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core cpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap xsaveopt xsavec xgetbv1 arat npt nrip_save

内存信息:
# free -h
              total used free shared buff/cache available
Mem: 15G 614M 13G 12M 801M 14G
Swap: 4.0G 0B 4.0G

[重现步骤]:
1. reboot机器
2. dmesg -l err -T查询开机是否有err日志


[期望结果]:
开机启动后,dmesg日志中无error日志

[实际结果]:
开机启动后,dmesg日志中有error日志
标签没加标签.

活动

Shiloong

2021-10-28 14:46

开发人员   ~0000551

这个看起来与 iptable 相关, 需要给更多的信息, 上下文关联 log.
也请 @无牙 帮忙一起看看.

dust-li

2021-11-01 17:18

报告者   ~0000568

问题是 iptables 在启动的时候会尝试加载 bpfilter, bpfilter 会尝试启动 bpfilter_umh,然后内核会尝试通过一个 pipe 和用户态的进程进行通信,失败在从 pipe 上读取数据的过程中。

而最后失败的原因是,anolis 7.7 上面,audit 打开了 selinux,selinux 在启动阶段 block 了 kernel 从 piple 里面读取数据,认为这个 pipe 的文件没有权限,最终返回 EACESS。

失败的调用链是:
__bpfilter_process_sockopt()
    --> kernel_read()
        --> vfs_read()
            --> rw_verify_area()
                --> security_file_permission()
                    --> selinux_file_permission()
                        --> selinux_revalidate_file_permission()
                            --> file_has_perm()
                                --> avc_has_perm()
                                    --> avc_has_perm_noaudit()
                                         --> avc_denied()



对比了 alibaba cloud linux2 和 anolis 7.7,原因在于 anolis 7.7 没有关闭 selinux,而 alibaba cloud linux 2 关掉了 selinux。

=== Anolis 7.7 ===
[ 0.765467] audit: type=1404 audit(1635753983.640:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=1 old-enabled=1 lsm=selinux res=1
[ 0.851271] SELinux: Permission getrlimit in class process not defined in policy.
[ 0.852581] SELinux: Class xdp_socket not defined in policy.
[ 0.853415] SELinux: the above unknown classes and permissions will be allowed
[ 0.854401] SELinux: policy capability network_peer_controls=1
[ 0.855198] SELinux: policy capability open_perms=1
[ 0.855872] SELinux: policy capability extended_socket_class=1
[ 0.856692] SELinux: policy capability always_check_network=0
[ 0.857492] SELinux: policy capability cgroup_seclabel=1
[ 0.858235] SELinux: policy capability nnp_nosuid_transition=1
[ 0.876521] audit: type=1403 audit(1635753983.751:3): auid=4294967295 ses=4294967295 lsm=selinux res=1


=== Alibaba cloud linux2 ===
[ 1.552653] SELinux: Disabled at runtime.
[ 1.575991] audit: type=1404 audit(1635778826.146:2): enforcing=0 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=0 old-enabled=1 lsm=selinux res=1


selinux 在启动阶段将 pipe 的权限给限制了,导致 bpfilter 读不出来,报错。

对比两者启动日志,可以看出, audit 的配置应该存在差异,导致 anolis 后面打开了 selinux 最终引起了该问题。

=== Anolis ===
[ 0.276350] audit: initializing netlink subsys (disabled)
[ 0.277124] audit: type=2000 audit(1635753983.372:1): state=initialized audit_enabled=0 res=1
[ 0.765467] audit: type=1404 audit(1635753983.640:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=1 old-enabled=1 lsm=selinux res=1
[ 0.876521] audit: type=1403 audit(1635753983.751:3): auid=4294967295 ses=4294967295 lsm=selinux res=1
[ 0.915877] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)

=== Alibaba cloud linux ===
[ 0.184471] audit: initializing netlink subsys (disabled)
[ 0.184989] audit: type=2000 audit(1635750119.588:1): state=initialized audit_enabled=0 res=1
[ 0.941037] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[ 1.575991] audit: type=1404 audit(1635778826.146:2): enforcing=0 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=0 old-enabled=1 lsm=selinux res=1

Shiloong

2021-11-01 17:24

开发人员   ~0000569

@jacbo_wang @geliwei_ali 帮忙看一下 Anolis7.7 的 baseOS 跟 Alinux2 相比 是不是 SELinux 默认打开.
不过总体看, 我理解不 block release. 这个目前看只影响启动时的 log 检查, 没有功能上的影响. @dust_li

问题历史

日期 用户名 字段 更改
2021-10-13 15:46 wb-wpp899309 新建问题
2021-10-19 12:33 geliwei-ali 分派给 => Shiloong
2021-10-19 12:33 geliwei-ali 状态 新建 => 已分配
2021-10-28 14:46 Shiloong 注释已添加: 0000551
2021-11-01 17:18 dust-li 注释已添加: 0000568
2021-11-01 17:24 Shiloong 注释已添加: 0000569
2021-11-17 10:41 Shiloong 分派给 Shiloong => XuanZhuo
2021-11-17 10:41 Shiloong 优先级 中 => 低