thiết bị ome
Trang 1Fault Management-
Customer Visible Logs
Standard Release 6.0 Issue 1 September 2009
What’s inside
Introduction to customer visible logs
Domain Optical Controller (DOC) logs
Trang 2Networks, the holder is granted no rights to use the information contained herein and this document shall not be published, copied, produced or reproduced, modified, translated, compiled, distributed, displayed or transmitted, in whole or part, in any form or media.
This information is provided “as is”, and Nortel Networks does not make or provide any warranty of any kind, expressed or implied, including any implied warranties of merchantability, non-infringement of third party intellectual property rights, and fitness for a
Trang 3Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Trang 5Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
The following sections detail what’s new in Fault Management - Customer Visible Logs, 323-1851-840 Standard Issue 1 for Release 6.0
New features
No new logs have been added in this issue
Trang 7Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
1-This section provides an introduction to the use of customer visible logs in troubleshooting failures related to the Domain Optical Controller
Trang 8alarms to signal fault conditions These alarms include:
Faults that result in one of these alarms prevent DOC from continuing a re-optimization as this could result in worse performance conditions for surviving traffic
DOC maintains logs detailing any failure of previously executed commands that can result in the above listed alarms The DOC Customer Visible Logs are retrievable from the Site Manager DOC application and the “Logs” button.The DOC logs are used to determine the fault responsible for the DOC alarm
on the network element The procedures contained in this guide outline the steps required to clear the faults based on the content of the logs Once the fault is cleared DOC can re-attempt the channel action
DOC Log Description
The DOC logs are displayed by opening the Configuration -> Photonic Services -> Domain Optical Controller (DOC) tab in Site Manager for a network element containing a DOC site Once a DOC instance is selected the user can Click the "Logs" button to view the CVLs
In the case of any DOC Failure the DOC Logs should be retrieved and saved for investigation before any troubleshooting steps are taken DOC logs are updated when a DOC action failure occurs The logs remain until either the next failure or when the "Clear DOC Logs" button is selected in Site Manager.DOC Logs report details for 5 sections:
Trang 9Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
The topology logs outline the system and channel topologies for the DOC, Network and Section The Command logs outline the steps taken, and the faults detected, that resulted in the alarm being raised
For details on Domain and Sections see the Photonic services - Photonic advanced optical control section of the OME 6500 Planning Guide (Part 2 of 3)
DOC Log Format
An example of the DOC Log window is shown below The format of the DOC logs are also described in this section
***************************************************************************************
***************************************************************************************
Trang 10Logs for <TID-SHELF>
***************************************************************************************
* Domain Optical Control Topology Log
*
***************************************************************************************DOC topology at <YYYY-MM-DD> <hh:mm:ss>
DOC Site <TID>-<SHELF>
Configuration: <Configuration>
List of SOC SitesTID-Shelf
========
<TID>-<SHELF>-<PATH ID> (repeated for all SOC Sites)
Network Channel AID Optimization State SHELF (Add & Drop)NC-<SHELF>-<SITEID>-<Remote Shelf>-<Remote Path>-<Wavelength>
<Optimization State of Channel> <TID of Add Shelf>-<SHELF>-<PATH ID>
<TID of Drop Shelf>-<SHELF>-<PATH ID> (repeated for all defined channels)
***************************************************************************************
* Domain Optical Control Command Log
*
***************************************************************************************
Trang 11Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
<YYYY-MM-DD> <hh:mm:ss> Section Topology log (repeated for all logs)
DOC Log Troubleshooting Documents
The procedures contained in this guide contain details on how to troubleshoot failures based on the contents of the DOC Logs The procedures are presented in the following format:
Log text - The text presented to the user in the DOC LogDescription/Probable cause - Outlines the cause of the faultTroubleshooting information - Details on resolving the fault
Trang 12describing directions in the photonic network:
Upstream: Components that exists prior to the current point in the optical traffic path
Downstream: Components that exists after the current point in the optical traffic path
Example
TX -CMD44(1) WSS(1) -MLA(1) MLA(2) -WSS(2) CMD44(2) RXThe TX is upstream of CMD44(1)
CMD44(2) is downstream of WSS(2)
Trang 13Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
2-This section provides detailed information on DOC logs That information is to
be used for troubleshooting purposes
Trang 14Topology Routing Failure - Routing message failed to sent to:
<TID>-<SHELF>, rem_path <PATH>
A failure occurred while trying to send Routing message to the next node with
<TID>-<SHELF> and rem_path <PATH> The probable cause is a communication error between network elements in the section
A retry mechanism will ensure that routing message is eventually sent once the error condition is resolved
Troubleshooting information
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Trang 15Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Update remote section routing request to <TID>-<SHELF> timed out, rem_path <PATH>
If that is the case then the probable cause is a communication error between network elements in the section
A retry mechanism will ensure that routing message is eventually sent once the error condition is resolved
Troubleshooting information
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
CVL 65549, 65581 might be seen after this one is generated
Trang 16"Network Topology Build Failure - ""DOC incorrectly provisioned - Consult
Description/Probable cause
The provisioning data for the Optical Domain is incorrect The following rules must be observed when provisioning an Optical Domain:
for linear systems
network path is valid when the OTSs’ Tx path values in a direction have the same parity, i.e all odd or all even values
Troubleshooting information
Site Manager
• Click on "Overlays " and select the "DOC indicator" and the "OSID indicator" options Click Ok
if it is a linear configuration
Trang 17Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Topology" tab in the Visualization application
an OTS instance" procedure to change the "DOC Indicator" parameter value in the OTS
re-provisioned with the correct value
TL1
OTS parameters:
RTRV-OTS::OTS-<SHELF>-<INSTANCE>:CTAG;
value in the OTS:
ED-OTS::OTS-<SHELF>-<INSTANCE>:CTAG:::DOCIND=<docind>;
<docind>: Y to set as a DOC instance, N to not site as a DOC instance
delete the OTS and re-provision it with the correct Tx path value
—end—
Additional information
None
Trang 18Network Topology Build Failure - Refer to Section Logs at <TID>-<SHELF>.
communication issue or a provisioning error
Troubleshooting information
Site Manager
Login to the node hosting the DOC for which this log was generated
Launch the Configuration -> Domain Optical Controller (DOC) application, and select the DOC instance from the list that is managing the optical section starting at TID-SHELF
Click on the "DOC Logs" button In the DOC Logs window, scroll domain to the
"Section Topology Logs" section and examine the detailed logs for the section TID-SHELF
Refer to the troubleshooting section of the detailed logs for more information
on how to resolve the problem
TL1
Login to the node hosting the DOC for which this log was generated
Use the RTRV-DOC-LOG command to retrieve the DOC logs and examine the detailed logs for the section TID-SHELF
Refer to the troubleshooting section of the detailed logs for more information
on how to resolve the problem
Trang 19Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Network Topology Build Failure - Number of sections in network exceeds the
<Maximum number of sections allowed in a network> allowed
Troubleshooting information
exceed the maximum limit
—end—
Additional information
None
Trang 20Network Topology Build Failure
A failure occurred while trying to build the network topology
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
Trang 21Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Network Topology Build Failure - Timeout of RoutingSet due to Invalid Path - OP_NECHT
Description/Probable cause
The topology application failed to set the routing information on one or more nodes in the optical domain This log can be generated when there is a problem with inter-node communication or when the domain is not provisioned incorrectly
Troubleshooting information
Site Manager
might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
are alarms that indicate a communication failure, such as "OSC Loss of Signal" or "Adjacency Far End Not Discovered" Refer to the alarm troubleshooting section for details on clearing any alarm present
release which is running all nodes in the network If any node is running
an older release, proceed with the appropriate upgrade procedure to ensure all nodes are running the same release
equipment and facilities are present and in-service for all nodes in the network
more than two DOC sites present in each domain
Trang 22are alarms that indicate a communication failure, such as "OSC Loss of Signal" or "Adjacency Far End Not Discovered" Refer to the alarm troubleshooting section for details on clearing any alarm present
is running all nodes in the network If any node is running an older release, proceed with the appropriate upgrade procedure to ensure all nodes are running the same release
equipment and facilities are present and in-service for all nodes in the domain
domain are listed as expected
DOC sites present in each domain
Additional information
None
Trang 23Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Network Topology Build Failure - Timeout of RoutingSet due to Invalid Path - OP_OSCT
Description/Probable cause
The topology application failed to set the routing information on one or more nodes in the optical domain This log can be generated when:
Troubleshooting information
Site Manager
might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
are alarms that indicate a communication failure, such as "OSC Loss of Signal" or "Adjacency Far End Not Discovered" Refer to the alarm troubleshooting section for details on clearing any alarm present
release which is running all nodes in the network If any node is running
an older release, proceed with the appropriate upgrade procedure to ensure all nodes are running the same release
Trang 24more than two DOC sites present in each domain
displayed as expected
TL1
are alarms that indicate a communication failure, such as "OSC Loss of Signal" or "Adjacency Far End Not Discovered" Refer to the alarm troubleshooting section for details on clearing any alarm present
is running all nodes in the network If any node is running an older release, proceed with the appropriate upgrade procedure to ensure all nodes are running the same release
equipment and facilities are present and in-service for all nodes in the domain
domain are listed as expected
DOC sites present in each domain
Additional information
None
Trang 25Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Network Topology Build Failure - Local section data is invalid or unavailable
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
CVL 65541 might be seen along with this one
Trang 26Network Topology Build Failure - Network Path could not be resolved by NCT
at <TID>-<SHELF> rem_path <PATH> Consult logs and Network Viewer
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Trang 27Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Network Topology Build Failure - Network Path could not be resolved by NCT
at <TID>-<SHELF> Consult logs and Network Viewer
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
CVL 65540 might be seen before this one is generated
Trang 28Sectional Topology Build Failure - Local shelf wavelength data is invalid or unavailable at <TID>-<SHELF>
available, a condition might exists on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Trang 29Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Sectional Topology Build Failure - Number of shelves in section starting with
<TID>-<SHELF> exceeds <Maximum number of shelves allowed in a section>
Troubleshooting information
number of shelves in it does not exceed the maximum limit
—end—
Additional information
None
Trang 30Sectional Topology Build Failure - Sectional Path could not be resolved by OSCT at <TID>-<SHELF> rem_path <PATH> Consult section logs.
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
Trang 31Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Sectional Topology Build Failure - Sectional Path could not be resolved by OSCT at <TID>-<SHELF> Consult section logs
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
None
Trang 32Topology Build Failure - Comms failure receiving from <TID>-<SHELF> rem_path <PATH>.
communication problem between network elements in the section
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
Trang 33Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Topology Build Failure - Comms failure receiving from <TID>-<SHELF>, rem_path <PATH>
communication problem between network elements in the section
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
None
Trang 34Topology Build Failure - Failed to open TCP connection with <TID>-<SHELF> send_path <PATH>.
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
Trang 35Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Topology Build Failure - Failed to receive TCP data response from
<TID>-<SHELF>, rem_path <PATH>
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
None
Trang 36Topology Build Failure - Failed to send TCP data request from
<TID1>-<SHELF1> to <TID2>-<SHELF2>, rem_path <PATH>
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
Trang 37Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Topology Build Failure - Invalid TCP message data from <TID>-SHELF> rem_path <PATH>
communication problem between network elements in the section
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
None
Trang 38Topology Build Failure - Invalid TCP message data from <TID>-<SHELF>.
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Trang 39Optical Multiservice Edge 6500 Fault Management - Customer Visible Logs
Release 6.0 Sep 2009
Copyright 2004-2009, Nortel Networks
Topology Build Failure - TCP/UDP message version mismatch detected by
<TID>-<SHELF> rem_path <PATH>
A retry mechanism will ensure that the channel topology is eventually discovered once the error condition is resolved
Troubleshooting information
available, a condition might exist on the network which is preventing normal communication between network elements This should be addressed using network troubleshooting techniques
clearing procedures
any NE is running an older release, proceed with the appropriate upgrade procedure to upgrade all NEs to the same software release
nodes in the network
Visualization tool (Network View) If the Visualization tool is not available issue the command RTRV-TOPO-OST and verify that all nodes in the network are listed as expected
—end—
Additional information
None
Trang 40Topology Build Failure - TCP/UDP message version mismatch detected by
<TID>-<SHELF>, rem_path <PATH>