How to find internet is working from the Private subnet

1) curl ipinfo.io

2) ping 

Different Types of Gateways in Cloud

Taken from net. 

Core Linux Kernel Components

1️⃣ Process Management

Handles creation, scheduling, and termination of processes.

• Creates processes (fork, exec) 

• Schedules CPU time (scheduler) 

• Context switching between processes 

👉 Example:

• When you run top, multiple processes compete for CPU → kernel decides who runs 

 

2️⃣ Memory Management

Controls how memory (RAM) is used.

• Virtual memory (each process gets its own space) 

• Paging & swapping 

• Memory allocation & deallocation 

👉 Key concept:

• Virtual Memory lets processes think they have large continuous memory 

 

3️⃣ File System (VFS Layer)

Manages files and storage devices.

• Provides a unified interface via Virtual File System (VFS) 

• Supports multiple filesystems: 

o ext4 

o xfs 

o btrfs 

👉 Example:

• /home, /etc, /var all handled through VFS abstraction 

 

4️⃣ Device Drivers

Interface between hardware and kernel.

• Disk drivers 

• Network card drivers 

• USB drivers 

👉 Without drivers:

• OS cannot talk to hardware 

 

5️⃣ Networking Stack

Handles all networking operations.

• TCP/IP protocol implementation 

• Packet routing 

• Socket interface 

👉 Example:

• When you ping or curl, kernel networking stack handles packets 

 

6️⃣ System Call Interface (SCI)

Bridge between user space and kernel space.

• Programs request kernel services via system calls 

• Examples: 

o read() 

o write() 

o open() 

👉 Think:

System calls = controlled entry points into kernel

 

7️⃣ Security (LSM Framework)

Provides security mechanisms.

• SELinux 

• AppArmor 

• Capabilities 

👉 Example:

• Restrict what processes can access 

 

8️⃣ Inter-Process Communication (IPC)

Allows processes to communicate.

• Pipes 

• Shared memory 

• Message queues 

• Signals 

 

9️⃣ Resource Management (Cgroups)

Controls resource usage per process/group.

• CPU limits 

• Memory limits 

• I/O limits 

👉 Uses:

• Containers (Docker) 

 

🔟 Namespaces (Isolation)

Provides isolation between processes.

• PID, NET, MNT, USER, etc. 

👉 Used heavily in containers

 

🧩 How It All Fits Together

User Applications

       ↓

System Calls

       ↓

---------------------

|   Linux Kernel    |

|-------------------|

| Process Mgmt      |

| Memory Mgmt       |

| File System       |

| Networking        |

| Drivers           |

| Security          |

---------------------

       ↓

Hardware

 

⚡ Quick Real-Life Flow

Example: cat file.txt

1. User runs command 

2. System call → open(), read() 

3. Kernel: 

o Checks permissions (security) 

o Uses filesystem to locate file 

o Uses disk driver to read data 

4. Data returned to user 

 

🧠 Key Takeaways

• Kernel = brain of the OS 

• Everything goes through it: 

o CPU 

o Memory 

o Disk 

o Network 

• Features like namespaces and cgroups enable modern containers 

How to fix the file system issue of file fs/xfs/libxfs/xfs_alloc.c

How to fix 1963 of file fs/xfs/libxfs/xfs_alloc.c

[FAILED] Failed to start RDMA Node Description Daemon.

See 'systemctl status rdma-ndd.service' fo[   19.903854] XFS (dm-6): Internal error ltbno + ltlen > bno at line 1963 of file fs/xfs/libxfs/xfs_alloc.c.  Caller xfs_free_ag_extent+0x42b/0x8b0 [xfs]

r details.

[   19.923764] XFS (dm-6): Corruption detected. Unmount and run xfs_repair

[   19.932503] XFS (dm-6): Internal error xfs_trans_cancel at line 954 of file fs/xfs/xfs_trans.c.  Caller xfs_efi_item_recover+0x229/0x250 [xfs]

[   19.950413] XFS (dm-6): Corruption of in-memory data (0x8) detected at xfs_trans_cancel+0x103/0x110 [xfs] (fs/xfs/xfs_trans.c:955).  Shutting down filesystem.

[   19.970172] XFS (dm-6): Please unmount the filesystem and rectify the problem(s)

[   19.981270] XFS (dm-6): Failed to recover intents

[FAILED] Failed to mount /sysroot.

See 'systemctl status sysroot.mount' for details.

[DEPEND] Dependency failed for Initrd Root File System.

[DEPEND] Dependency failed for Reload Configuration from the Real Root.

         Stopping Hardware RNG Entropy Gatherer Daemon...

         Starting Setup Virtual Console...

[  OK  ] Stopped dracut pre-mount hook.

[  OK  ] Stopped dracut initqueue hook.

[  OK  ] Stopped dracut pre-trigger hook.

[  OK  ] Stopped dracut pre-udev hook.

[  OK  ] Stopped dracut cmdline hook.

[  OK  ] Reached target Initrd File Systems.

[  OK  ] Started Setup Virtual Console.

[  OK  ] Stopped Hardware RNG Entropy Gatherer Daemon.

[  OK  ] Stopped target Basic System.

[  OK  ] Stopped target System Initialization.

[  OK  ] Stopped RDMA Node Description Daemon.

[FAILED] Failed to start RDMA Node Description Daemon.

See 'systemctl status rdma-ndd.service' for details.

[  OK  ] Started Emergency Shell.

[  OK  ] Reached target Emergency Mode.

Generating "/run/initramfs/rdsosreport.txt"

Entering emergency mode. Exit the shell to continue.

Type "journalctl" to view system logs.

You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot

after mounting them and attach it to a bug report.

:/#

[   19.767320] sukhoi: XFS (dm-6): Internal error ltbno + ltlen > bno at line 1963 of file fs/xfs/libxfs/xfs_alloc.c.  Caller xfs_free_ag_extent+0x42b/0x8b0 [xfs]

[   19.782929] sukhoi: CPU: 93 PID: 1934 Comm: mount Not tainted 5.15.0-308.179.6.7.el8uek.x86_64 #2

[   19.782932] sukhoi: Hardware name: Oracle Corporation ORACLE SERVER X8-2/ASM,MB,X8-2, BIOS 51150100 03/24/2025

[   19.782934] sukhoi: Call Trace:

[   19.782936] sukhoi:  <TASK>

[   19.782938] sukhoi:  ? xfs_free_ag_extent+0x42b/0x8b0 [xfs]

[   19.782995] sukhoi:  dump_stack_lvl+0x46/0x69

[   19.783000] sukhoi:  xfs_corruption_error+0x83/0x90 [xfs]

[   19.783057] sukhoi:  ? xfs_free_ag_extent+0x42b/0x8b0 [xfs]

[   19.783104] sukhoi:  xfs_free_ag_extent+0x459/0x8b0 [xfs]

[   19.783150] sukhoi:  ? xfs_free_ag_extent+0x42b/0x8b0 [xfs]

[   19.783197] sukhoi:  __xfs_free_extent+0xdc/0x1e0 [xfs]

[   19.783244] sukhoi:  xfs_trans_free_extent+0x3d/0xf0 [xfs]

[   19.783305] sukhoi:  xfs_efi_item_recover+0x219/0x250 [xfs]

[   19.783365] sukhoi:  ? xfs_buf_rele+0x53/0x2c0 [xfs]

[   19.783421] sukhoi:  xlog_recover_process_intents+0xa3/0x310 [xfs]

[   19.783482] sukhoi:  ? xfs_mod_fdblocks+0xb1/0x1a0 [xfs]

[   19.783539] sukhoi:  ? __xfs_ag_resv_init+0x4a/0x140 [xfs]

[   19.783592] sukhoi:  xlog_recover_finish+0x13/0x120 [xfs]

[   19.783653] sukhoi:  ? xfs_fs_reserve_ag_blocks+0x5f/0xd0 [xfs]

[   19.783709] sukhoi:  xfs_log_mount_finish+0xdc/0x160 [xfs]

[   19.783768] sukhoi:  xfs_mountfs+0x4fb/0x870 [xfs]

[   19.783826] sukhoi:  xfs_fs_fill_super+0x4d1/0x890 [xfs]

[   19.783885] sukhoi:  ? xfs_fs_put_super+0x140/0x140 [xfs]

[   19.783943] sukhoi:  get_tree_bdev+0x17b/0x26c

[   19.783950] sukhoi:  vfs_get_tree+0x22/0xd8

[   19.783953] sukhoi:  do_new_mount+0x1a2/0x2ef

[   19.783959] sukhoi:  __x64_sys_mount+0x123/0x15a

[   19.783962] sukhoi:  do_syscall_64+0x35/0x87

[   19.783965] sukhoi:  entry_SYSCALL_64_after_hwframe+0x6e/0x0

[   19.783969] sukhoi: RIP: 0033:0x7fc9f7468dbe

[   19.783971] sukhoi: Code: 48 8b 0d cd 70 39 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 9a 70 39 00 f7 d8 64 89 01 48

[   19.783972] sukhoi: RSP: 002b:00007ffd5f51c0d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5

[   19.783974] sukhoi: RAX: ffffffffffffffda RBX: 00005572c1ff0ef0 RCX: 00007fc9f7468dbe

[   19.783975] sukhoi: RDX: 00005572c1ff87e0 RSI: 00005572c1ff10d0 RDI: 00005572c1ff1de0

[   19.783976] sukhoi: RBP: 00007fc9f82ec184 R08: 0000000000000000 R09: 00005572c1ff0014

[   19.783977] sukhoi: R10: 00000000c0ed0001 R11: 0000000000000246 R12: 0000000000000000

[   19.783978] sukhoi: R13: 00000000c0ed0001 R14: 00005572c1ff1de0 R15: 00005572c1ff87e0

[   19.783979] sukhoi:  </TASK>

[   19.783984] sukhoi: XFS (dm-6): Corruption detected. Unmount and run xfs_repair

[   19.791568] sukhoi: XFS (dm-6): Internal error xfs_trans_cancel at line 954 of file fs/xfs/xfs_trans.c.  Caller xfs_efi_item_recover+0x229/0x250 [xfs]

[   19.806291] sukhoi: CPU: 93 PID: 1934 Comm: mount Not tainted 5.15.0-308.179.6.7.el8uek.x86_64 #2

[   19.806293] sukhoi: Hardware name: Oracle Corporation ORACLE SERVER X8-2/ASM,MB,X8-2, BIOS 51150100 03/24/2025

[   19.806294] sukhoi: Call Trace:

[   19.806295] sukhoi:  <TASK>

[   19.806296] sukhoi:  dump_stack_lvl+0x46/0x69

[   19.806299] sukhoi:  xfs_trans_cancel+0xea/0x110 [xfs]

[   19.806379] sukhoi:  xfs_efi_item_recover+0x229/0x250 [xfs]

[   19.806443] sukhoi:  xlog_recover_process_intents+0xa3/0x310 [xfs]

[   19.806507] sukhoi:  ? xfs_mod_fdblocks+0xb1/0x1a0 [xfs]

[   19.806568] sukhoi:  ? __xfs_ag_resv_init+0x4a/0x140 [xfs]

[   19.806624] sukhoi:  xlog_recover_finish+0x13/0x120 [xfs]

[   19.806688] sukhoi:  ? xfs_fs_reserve_ag_blocks+0x5f/0xd0 [xfs]

[   19.806747] sukhoi:  xfs_log_mount_finish+0xdc/0x160 [xfs]

[   19.806810] sukhoi:  xfs_mountfs+0x4fb/0x870 [xfs]

[   19.806872] sukhoi:  xfs_fs_fill_super+0x4d1/0x890 [xfs]

[   19.806934] sukhoi:  ? xfs_fs_put_super+0x140/0x140 [xfs]

[   19.806995] sukhoi:  get_tree_bdev+0x17b/0x26c

[   19.806999] sukhoi:  vfs_get_tree+0x22/0xd8

[   19.807002] sukhoi:  do_new_mount+0x1a2/0x2ef

[   19.807005] sukhoi:  __x64_sys_mount+0x123/0x15a

[   19.807008] sukhoi:  do_syscall_64+0x35/0x87

[   19.807011] sukhoi:  entry_SYSCALL_64_after_hwframe+0x6e/0x0

[   19.807015] sukhoi: RIP: 0033:0x7fc9f7468dbe

[   19.807016] sukhoi: Code: 48 8b 0d cd 70 39 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 9a 70 39 00 f7 d8 64 89 01 48

[   19.807018] sukhoi: RSP: 002b:00007ffd5f51c0d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5

[   19.807019] sukhoi: RAX: ffffffffffffffda RBX: 00005572c1ff0ef0 RCX: 00007fc9f7468dbe

[   19.807020] sukhoi: RDX: 00005572c1ff87e0 RSI: 00005572c1ff10d0 RDI: 00005572c1ff1de0

[   19.807021] sukhoi: RBP: 00007fc9f82ec184 R08: 0000000000000000 R09: 00005572c1ff0014

[   19.807022] sukhoi: R10: 00000000c0ed0001 R11: 0000000000000246 R12: 0000000000000000

[   19.807023] sukhoi: R13: 00000000c0ed0001 R14: 00005572c1ff1de0 R15: 00005572c1ff87e0

[   19.807024] sukhoi:  </TASK>

[   19.807040] sukhoi: XFS (dm-6): Corruption of in-memory data (0x8) detected at xfs_trans_cancel+0x103/0x110 [xfs] (fs/xfs/xfs_trans.c:955).  Shutting down filesystem.

[   19.827652] sukhoi: XFS (dm-6): Please unmount the filesystem and rectify the problem(s)

[   19.838029] sukhoi: XFS (dm-6): Failed to recover intents

[   19.846003] sukhoi: XFS (dm-6): Ending recovery (logdev: internal)

[   19.846398] sukhoi: XFS (dm-6): log mount finish failed

[   15.979749] sukhoi mount[1934]: mount: /sysroot: mount(2) system call failed: Structure needs cleaning.

[   15.981015] sukhoi systemd[1]: sysroot.mount: Mount process exited, code=exited status=32

[   15.981055] sukhoi systemd[1]: sysroot.mount: Failed with result 'exit-code'.

[   15.981082] sukhoi systemd[1]: Failed to mount /sysroot.[10:01 PM]There is a file system corruption here , You can  work with Linux team to fix  OR reimage this . (edited) [10:01 PM][   19.807040] sukhoi: XFS (dm-6): Corruption of in-memory data (0x8) detected at xfs_trans_cancel+0x103/0x110 [xfs] (fs/xfs/xfs_trans.c:955).  Shutting down filesystem.

pre-mount:/# ls -al /dev/mapper

/sbin/xfs_repair -n /dev/mapper/VGExaDb-LVDbSys1

sbin/xfs_repair -L /dev/mapper/VGExaDb-LVDbSys1

/sbin/reboot -f

How to collect the ExaWatcher logs from the node.

/opt/oracle.ExaWatcher/GetExaWatcherResults.sh --from 2/13/2026_12:00:00 --to 2/14/2026_07:00:00

How to find which IB switch is running SM master

Run the following command on the IB switch sminfo

Kernel parameter to collect the VMcore to be added in /etc/sysctl.conf

kernel.hardlockup_panic=1 

kernel.hung_task_panic=1 

kernel.max_rcu_stall_to_panic=1 

kernel.panic_on_rcu_stall=1 

kernel.softlockup_panic=1

 kernel.unknown_nmi_panic=1