Get webhook notifications whenever Network & Infrastructure creates an incident, updates an incident, resolves an incident or changes a component status.
Following to http://status.ovh.net/?do=details&id=3534 we have had a vlan corruption on some N5
switch. Cisco worked on the bug
CSCuc37057 and a new version was released:
5.2.1.N1.2a
We will update a switch pdc1-17-n5
to validate the reparation of the vlan status.
Update(s):
Date: 2012-11-23 00:12:44 UTC p19-53/54-n5: done with a cutoff in some old internal services. a fex has crashed.
p19-54-n5# sh install all status
This is the log of last installation.
Need to perform cleanup of failed upgrade.
Install has failed. Return code 0x40930039 (aborting due to failed upgrade).
Please identify the cause of the failure, and try 'install all' again.
Reason: Reset Requested due to Fatal System Error
System version: 5.2(1)N1(2a)
Service: ISSU failure: 0x4093003B
Date: 2012-11-22 15:53:41 UTC vrack-1a/b-n5: doing
vrack-2a/b-n5: todo
Date: 2012-11-22 13:32:58 UTC arc-2a/b-n5: todo
omer-2a/b-n5: todo
Date: 2012-11-22 13:32:50 UTC 4 rese&aux de HG:
188.165.13.0/24: done
188.165.14.0/24: done
188.165.15.0/24: done
178.33.122.0/24: done
Date: 2012-11-22 13:32:39 UTC arc-1a/1b-n5: doing
Nov 22 13:49:44 arc-1a-n5 %VPC-2-VPC_ISSU_START: Peer vPC switch ISSU start, locking configuration
2012 Nov 22 13:53:05 arc-1a-n5 %SATCTRL-FEX100-2-SATCTRL_IMAGE: FEX100 Image update in progress.
2012 Nov 22 13:53:15 arc-1a-n5 %SATCTRL-FEX101-2-SATCTRL_IMAGE: FEX101 Image update in progress.
2012 Nov 22 13:53:26 arc-1a-n5 %SATCTRL-FEX102-2-SATCTRL_IMAGE: FEX102 Image update in progress.
2012 Nov 22 13:53:36 arc-1a-n5 %SATCTRL-FEX103-2-SATCTRL_IMAGE: FEX103 Image update in progress.
2012 Nov 22 13:53:46 arc-1a-n5 %SATCTRL-FEX104-2-SATCTRL_IMAGE: FEX104 Image update in progress.
2012 Nov 22 13:53:55 arc-1a-n5 %SATCTRL-FEX105-2-SATCTRL_IMAGE: FEX105 Image update in progress.
2012 Nov 22 13:54:05 arc-1a-n5 %SATCTRL-FEX106-2-SATCTRL_IMAGE: FEX106 Image update in progress.
2012 Nov 22 13:54:16 arc-1a-n5 %SATCTRL-FEX107-2-SATCTRL_IMAGE: FEX107 Image update in progress.
2012 Nov 22 13:54:26 arc-1a-n5 %SATCTRL-FEX108-2-SATCTRL_IMAGE: FEX108 Image update in progress.
2012 Nov 22 13:54:36 arc-1a-n5 %SATCTRL-FEX109-2-SATCTRL_IMAGE: FEX109 Image update in progress.
2012 Nov 22 13:54:46 arc-1a-n5 %SATCTRL-FEX110-2-SATCTRL_IMAGE: FEX110 Image update in progress.
2012 Nov 22 13:54:56 arc-1a-n5 %SATCTRL-FEX111-2-SATCTRL_IMAGE: FEX111 Image update in progress.
2012 Nov 22 13:57:42 arc-1a-n5 %SATCTRL-FEX101-2-SATCTRL_IMAGE: FEX101 Image update complete. Install pending
2012 Nov 22 13:58:21 arc-1a-n5 %SATCTRL-FEX100-2-SATCTRL_IMAGE: FEX100 Image update complete. Install pending
2012 Nov 22 13:59:44 arc-1a-n5 %SATCTRL-FEX103-2-SATCTRL_IMAGE: FEX103 Image update complete. Install pending
2012 Nov 22 14:00:09 arc-1a-n5 %SATCTRL-FEX104-2-SATCTRL_IMAGE: FEX104 Image update complete. Install pending
2012 Nov 22 14:00:15 arc-1a-n5 %SATCTRL-FEX102-2-SATCTRL_IMAGE: FEX102 Image update complete. Install pending
2012 Nov 22 14:00:19 arc-1a-n5 %SATCTRL-FEX106-2-SATCTRL_IMAGE: FEX106 Image update complete. Install pending
2012 Nov 22 14:00:51 arc-1a-n5 %SATCTRL-FEX107-2-SATCTRL_IMAGE: FEX107 Image update complete. Install pending
2012 Nov 22 14:00:51 arc-1a-n5 %SATCTRL-FEX105-2-SATCTRL_IMAGE: FEX105 Image update complete. Install pending
2012 Nov 22 14:01:01 arc-1a-n5 %SATCTRL-FEX111-2-SATCTRL_IMAGE: FEX111 Image update complete. Install pending
2012 Nov 22 14:01:06 arc-1a-n5 %SATCTRL-FEX110-2-SATCTRL_IMAGE: FEX110 Image update complete. Install pending
2012 Nov 22 14:01:09 arc-1a-n5 %SATCTRL-FEX109-2-SATCTRL_IMAGE: FEX109 Image update complete. Install pending
2012 Nov 22 14:01:13 arc-1a-n5 %SATCTRL-FEX108-2-SATCTRL_IMAGE: FEX108 Image update complete. Install pending
2012 Nov 22 14:02:17 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-100 On-line
2012 Nov 22 14:02:18 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 100 is online
2012 Nov 22 14:03:34 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-101 On-line
2012 Nov 22 14:03:34 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 101 is online
2012 Nov 22 14:04:50 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-102 On-line
2012 Nov 22 14:04:51 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 102 is online
2012 Nov 22 14:06:07 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-103 On-line
2012 Nov 22 14:06:08 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 103 is online
2012 Nov 22 14:07:23 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-104 On-line
2012 Nov 22 14:07:24 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 104 is online
2012 Nov 22 14:08:39 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-105 On-line
2012 Nov 22 14:08:40 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 105 is online
2012 Nov 22 14:09:56 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-106 On-line
2012 Nov 22 14:09:56 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 106 is online
2012 Nov 22 14:11:11 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-107 On-line
2012 Nov 22 14:11:12 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 107 is online
2012 Nov 22 14:12:27 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-108 On-line
2012 Nov 22 14:12:28 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 108 is online
2012 Nov 22 14:13:43 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-109 On-line
2012 Nov 22 14:13:43 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 109 is online
2012 Nov 22 14:14:59 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-110 On-line
2012 Nov 22 14:14:59 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 110 is online
2012 Nov 22 14:16:14 arc-1a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-111 On-line
2012 Nov 22 14:16:15 arc-1a-n5 %PFMA-2-FEX_STATUS: Fex 111 is online
2012 Nov 22 14:17:01 arc-1a-n5 %VPC-2-VPC_ISSU_END: Peer vPC switch ISSU end, unlocking configuration
2012 Nov 22 14:28:04 arc-1b-n5 %VPC-2-VPC_ISSU_START: Peer vPC switch ISSU start, locking configuration
2012 Nov 22 14:29:29 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is DOWN in vdc 1
2012 Nov 22 14:29:35 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is UP in vdc 1
2012 Nov 22 14:29:41 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is DOWN in vdc 1
2012 Nov 22 14:29:47 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is UP in vdc 1
2012 Nov 22 14:30:29 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is DOWN in vdc 1
2012 Nov 22 14:30:38 arc-1b-n5 %IM-5-IM_INTF_STATE: mgmt0 is UP in vdc 1
2012 Nov 22 14:31:40 arc-1b-n5 %VPC-2-VPC_ISSU_END: Peer vPC switch ISSU end, unlocking configuration
Date: 2012-11-22 13:32:22 UTC We continue with updates:
Pcc-104-n5: done
Pcc-105-n5: done
Pcc-106-n5: done
Pcc-107-n5: done
Pcc-108-n5: done
Pcc-109-n5: done
Pcc-110-n5: done
Pcc-111-n5: done
Pcc-112-n5: done
Pcc-113-n5: done
Date: 2012-10-29 10:17:00 UTC storage-s31a/b-n5: doing
Date: 2012-10-29 10:16:52 UTC storage-s27a/b-n5: done
omer-1a/b-n5: done
Date: 2012-10-29 09:32:42 UTC storage-s15a/b-n5
Date: 2012-10-26 10:08:22 UTC rps-1a/rps-1b
Date: 2012-10-26 10:08:00 UTC pdc1-12-n5
backup-1-n5
Date: 2012-10-26 09:40:25 UTC before (we see the status UNKNOWN
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------- ---------------------- ---------------------- ------------
1 system 5.2(1)N1(1) 5.2(1)N1(2a) yes
1 kickstart 5.2(1)N1(1) 5.2(1)N1(2a) yes
1 bios v1.5.0(11/30/10) v1.5.0(11/30/10) no
1 power-seq v1.2 v1.2 no