How to spilt the Mellnox 100 G port to 4 * 25 GB QSFP

How to spilt the 100 G port to 4 * 25 GB QSFP Login to Mellanox switch enable config t interface ethernet 1/9 shutdown interface ethernet 1/10 shutdown int ethernet 1/9 interface ethernet 1/9 module-type qstp-split-4 type yes show interface ethernet status show interface ethernet 1/9/1 write m exit

How to Find Exadata Rack Model (Eight/Quarter/Half ..)

From the DB nodes.


cd /opt/oracle.SupportTools/onecommand/

grep -i MACHINETYPES databasemachine.xml

 

[root@sukhoi/]# grep -i MACHINETYPES
                X4-2 Eighth Rack HP 1.2TB

 

From the cell nodes

 

[root@Sukhoi ~]# cellcli -e list cell attributes eighthrack

TRUE

 

 

Regards

Gurudatta N.R

How to check the IB cables is latched properly in DB/Storage nodes in Exadata .

for ib_cable in `ls /sys/class/net | grep ^ib`; do printf "$ib_cable: "; cat /sys/class/net/$ib_cable/carrier; done

The output should look similar to:

ib0: 1
ib1: 1

If anything other than "1" is reported, double check the cable connection.

Regards
Gurudatta N.R

Command to find the ILOM Ipaddress from the operating system

1) check ipmitool is installed

rpm -qa | grep ipmitool

ipmitool-1.8.12.4-0.el6.x86_64

If the ipmi rpm is installed you can run the following command

ipmitool lan print


[root@Sukhoi ~]#  ipmitool lan print

Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD
Auth Type Enable        : Callback : MD2 MD5 PASSWORD
                        : User     : MD2 MD5 PASSWORD
                        : Operator : MD2 MD5 PASSWORD
                        : Admin    : MD2 MD5 PASSWORD
                        : OEM      :
IP Address Source       : Static Address
IP Address              : XX.XX.XX.XX
Subnet Mask             : XXX.XXX.XXX.XX
MAC Address             : XX:XX:XX:XX:XX:XX
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Disabled, Gratuitous ARP Disabled
Gratituous ARP Intrvl   : 5.0 seconds
Default Gateway IP      : XX.XX.XX.XX
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3
Cipher Suite Priv Max   : XXXXXXXXXXXXXXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM

How F20 Modules are connected in X4270 M2 in Exadata.

<- -="" exadata="" nbsp=""><----- ------="" linux="" software=""><-- --="" active="" pcie="" risers=""><--- -----="" aura="" card="" nbsp="">

Exadata s/w  -> linux  -> scsi -> mptsas -------> riser 1 IDT --------> IOC1 (slot 1) ------> 4 FMODS
                                            |                       |
                                            |                        ----> IOC0 (slot 4) ------> 4 FMODS
                                            |
                                              ------> riser 2 IDT --------> IOC3 (slot 2) ------> 4 FMODS
                                                                    |
                                                                      ----> IOC2 (slot 5) ------> 4 FMODS
.

How to set the System_identifier in Engineered Systems.

At Linux prompt:

ipmitool sunoem cli 'set /SP system_identifier="Exadata Database Machine X3-2 AKXXXXXX”’

OR

In ilom:

set /SP system_identifier="Exadata Database Machine X3-2 AKXXXXXX"

Regards

Gurudatta N.R

How to check the critical Alert logs in Exadata Cell nodes.

for i in 1 2 3 4 5 6 7
do
ssh root@cell{$i} “cellcli –e list alerthistory |grep –I critical”
done

How to clear the Alert history in the Exadata.

[root@sinacel05 ~]# cellcli
CellCLI: Release 11.2.3.1.1 - Production on Tue Feb 09 16:11:29 SGT 2016

Copyright (c) 2007, 2011, Oracle.  All rights reserved.
Cell Efficiency Ratio: 1

CellCLI>   LIST ALERTHISTORY WHERE severity = 'critical'  AND examinedBy = '' DETAIL

        name:                   2_1
         alertMessage:           "Cell configuration check discovered the following problems:   Check Exadata configuration via ipconf utility Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf Error. Exadata configuration file not found /opt/oracle.cellos/cell.conf [INFO] The ipconf check may generate a failure for temporary inability to reach NTP or DNS server. You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] As root user run /usr/local/bin/ipconf -verify -semantic to verify consistent network configurations."
         alertSequenceID:        2
         alertShortName:         Software
         alertType:              Stateful
         beginTime:              2015-10-06T16:10:23+08:00
         endTime:                2015-10-06T16:24:17+08:00
         examinedBy:
         metricObjectName:       checkconfig
         notificationState:      0
         sequenceBeginTime:      2015-10-06T16:10:23+08:00
         severity:               critical
         alertAction:            "Correct the configuration problems. Then run cellcli command:   ALTER CELL VALIDATE CONFIGURATION   Verify that the new configuration is correct."

         name:                   2_2
         alertMessage:           "Cell configuration check discovered the following problems:   Check Exadata configuration via ipconf utility Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf Checking NTP server on 192.168.1.254                                                             : FAILED Error. Overall status of verification of Exadata configuration file: FAILED [INFO] The ipconf check may generate a failure for temporary inability to reach NTP or DNS server. You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] As root user run /usr/local/bin/ipconf -verify -semantic to verify consistent network configurations."
         alertSequenceID:        2
         alertShortName:         Software
         alertType:              Stateful
         beginTime:              2015-10-06T16:24:17+08:00
         endTime:                2015-10-20T19:43:44+08:00
         examinedBy:
         metricObjectName:       checkconfig
         notificationState:      0
         sequenceBeginTime:      2015-10-06T16:10:23+08:00
         severity:               critical
         alertAction:            "Correct the configuration problems. Then run cellcli command:   ALTER CELL VALIDATE CONFIGURATION   Verify that the new configuration is correct."

         name:                   2_3
  alertMessage:           "Cell configuration check discovered the following problems:   Check Exadata configuration via ipconf utility Verifying of Exadata configuration file /opt/oracle.cellos/cell.conf Checking NTP server on 192.168.1.254                                                             : FAILED Error. Overall status of verification of Exadata configuration file: FAILED [INFO] The ipconf check may generate a failure for temporary inability to reach NTP or DNS server. You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] You may ignore this alert, if the NTP or DNS servers are valid and available. [INFO] As root user run /usr/local/bin/ipconf -verify -semantic to verify consistent network configurations.  The hardware and firmware are not supported. See details below  [PCISlot:HBA:LSIModel:LSIhw:MPThw:LSIfw:MPTBios:DOM:OSDevice:DOMMake:DOMModel:DOMfw:CountAuraCountDOM] Requires:  AllSlots_AllHBAs SAS1068E B3orC0 105 011b5c00 06.26.00.00 AllDOMs_NotApplicable MARVELL SD88SA02 D20Y  4_16   Found:  2 3 SAS1068E B3orC0 105 011b5c00 06.26.00.00 1 /dev/sdv MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  2 3 SAS1068E B3orC0 105 011b5c00 06.26.00.00 2 /dev/sdw MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  2 3 SAS1068E B3orC0 105 011b5c00 06.26.00.00 3 /dev/sdx MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  2 3 SAS1068E B3orC0 105 011b5c00 06.26.00.00 4 /dev/sdy MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  4 1 SAS1068E B3orC0 105 011b5c00 06.26.00.00 1 /dev/sdn MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  4 1 SAS1068E B3orC0 105 011b5c00 06.26.00.00 2 /dev/sdo MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  4 1 SAS1068E B3orC0 105 011b5c00 06.26.00.00 3 /dev/sdp MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  4 1 SAS1068E B3orC0 105 011b5c00 06.26.00.00 4 /dev/sdq MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  5 2 SAS1068E B3orC0 105 011b5c00 06.26.00.00 1 /dev/sdr MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  5 2 SAS1068E B3orC0 105 011b5c00 06.26.00.00 2 /dev/sds MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  5 2 SAS1068E B3orC0 105 011b5c00 06.26.00.00 3 /dev/sdt MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16  5 2 SAS1068E B3orC0 105 011b5c00 06.26.00.00 4 /dev/sdu MARVELL SD88SA02 D20Y DOMS_found_12_but_expected_16   The hardware and firmware are not supported. See details above"

CellCLI> drop alerthistory all ======> command to clear the Alert history

Alert 2_1 successfully dropped
Alert 2_2 successfully dropped
Alert 2_3 successfully dropped
Alert 2_4 successfully dropped
Alert 10_1 successfully dropped
Alert 10_2 successfully dropped

CellCLI> LIST ALERTHISTORY WHERE severity = 'critical'



CellCLI> drop alerthistory all

Alert 2_1 successfully dropped
Alert 2_2 successfully dropped
Alert 6_1 successfully dropped
Alert 6_2 successfully dropped

CellCLI> LIST ALERTHISTORY WHERE severity = 'critical'

CellCLI>




Regards
Gurudatta N.R

Cellcli commands used in Oracle Exadata

# cellcli
# cellcli -e list cell
CellCLI> list cell
CellCLI> list cell detail
CellCLI> list cell attributes all
CellCLI> list cell attributes rsStatus
CellCLI> list physicaldisk
CellCLI> list physicaldisk detail
CellCLI> list physicaldisk 20:3
CellCLI> list physicaldisk 20:11 detail
CellCLI> list physicaldisk attributes all
CellCLI> list physicaldisk attributes name, id, slotnumber
CellCLI> list physicaldisk attributes name, disktype, makemodel, physicalrpm, physicalport, status
CellCLI> list physicaldisk attributes name, disktype, errCmdTimeoutCount, errHardReadCount, errHardWriteCount
CellCLI> list physicaldisk where diskType='Flashdisk'
CellCLI> list physicaldisk attributes name, id, slotnumber where disktype="flashdisk" and status != "not present"
CellCLI> list physicaldisk attributes name, physicalInterface, physicalInsertTime where disktype = 'Harddisk'
CellCLI> list physicaldisk where diskType=flashdisk and status='poor performance' detail
CellCLI> describe cell
CellCLI> describe physicaldisk
CellCLI> describe lun
CellCLI> describe celldisk
CellCLI> describe griddisk
CellCLI> describe flashcache
CellCLI> describe flashcachecontent
CellCLI> describe metriccurrent
CellCLI> describe metricdefinition
CellCLI> describe metrichistory

CellCLI> list lun - Will display LUN information
CellCLI> list lun detail
CellCLI> list lun 0_8 detail
CellCLI> list lun attributes all
CellCLI> list lun attributes name, cellDisk, raidLevel, status
CellCLI> list lun where disktype=flashdisk

CellCLI> list celldisk display cell disks information
CellCLI> list celldisk detail
CellCLI> list celldisk FD_01_cell07
CellCLI> list celldisk FD_01_cell13 detail
CellCLI> list celldisk attributes all
CellCLI> list celldisk attributes name, devicePartition
CellCLI> list celldisk attributes name, devicePartition where size>20G
CellCLI> list celldisk attributes name,interleaving where disktype=harddisk

CellCLI> list griddisk grid disks information
CellCLI> list griddisk detail
CellCLI> list griddisk DG_03_cell19 detail
CellCLI> list griddisk attributes all
CellCLI> list griddisk attributes name, size
CellCLI> list griddisk attributes name, cellDisk, diskType
CellCLI> list griddisk attributes name, ASMDeactivationOutcome, ASMModeStatus --- describe command does not show these two attributes
CellCLI> list griddisk attributes name,cellDisk,status where size=476.546875G
CellCLI> list griddisk attributes name where asmdeactivationoutcome != 'Yes'
CellCLI> list flashcache - display flash cache information
CellCLI> list flashcache detail
CellCLI> list flashcache attributes all
CellCLI> list flashcache attributes degradedCelldisks

CellCLI> create celldisk all harddisk
CellCLI> create celldisk all
CellCLI> create celldisk all harddisk interleaving='normal_redundancy'
interleaving -- none(default), normal_redundancy or high_redundancy
CellCLI> create celldisk all flashdisk

CellCLI> create griddisk RECO_CD_10_cell10 celldisk=CD_11_cell10
CellCLI> create griddisk RECO_CD_09_cell10 celldisk=CD_09_cell10 size=100M
CellCLI> create griddisk all prefix RECO
CellCLI> create griddisk all flashdisk prefix FLASH
CellCLI> create griddisk all harddisk prefix HARD
CellCLI> create griddisk all harddisk prefix='data', size='270g'
CellCLI> create griddisk all prefix='data', size='300g'
CellCLI> create griddisk all prefix='redo', size='150g'
CellCLI> create griddisk all harddisk prefix=systemdg
CellCLI> create flashcache celldisk='FD_00_cell10'
CellCLI> create flashcache all
CellCLI> create flashcache all size=365.25G in flash F20 Cad.



Regards
Gurudatta N.R

Common Terminology used in Exadata.

IB - InfiniBand
ILOM - Integrated Lights Out Manager
IPMI - Intelligent Platform Management Interface
LUN - Logical Unit Number
HARD - Hardware Assisted Resilient Data
RDS - Reliable Data Socket protocol
iDB - Intelligent Database protocol
UDMs - User Defined Metrics
IOPS - IOs Per Second
OEL - Oracle Enterprise Linux
SNMP - Simple Network Management Protocol
MS - Management Server
RS - Restart Server
EM - Enterprise Manager
EMGC - Enterprise Manager Grid Control
HCA - Host Controller Adapter
HBA - Host Bus Adapter
SM - Subnet Manager
DBRM - Database Resource Manager
SATA - Serial Advanced Technology Attachment
SAS - Serial Attached SCSI
CU - Compression Unit
PDU - Power Distribution Unit
HCC - Hybrid Columnar Compression

Source: Net

Regards
Gurudatta N.R