XGS4600 - SFP+ Transceivers stop transmitting light

Bash
Bash Posts: 8
Friend Collector First Comment
edited August 2022 in Switch
We are having an issue where both 10Gb and 1Gbtransceivers stop transmitting light at seemingly random times. This can only be fixed by unplugging and replugging in the transceiver. When looking at the statistics we get the following;

stack# show interfaces transceiver 1/30
  Transceiver Information

Port : 1/30 (SFP)
Vendor : Fibrestore
Part Number : DWDM-SFP10G-80
Serial Number : J0609190156
Revision : A
Date Code : 2016-09-07
Transceiver : 10GBASE-LR
Calibration : Internal

++ : high alarm, + : high warn, - : low warn, -- : low alarm.
neg inf : negative infinity(0mW).

               Current  High Alarm   High Warn    Low Warn   Low Alarm
                         Threshold   Threshold   Threshold   Threshold
--------------- -------------- ----------- ----------- ----------- -----------
Temperature(C) ++       25.00        0.00        0.00        0.00        0.00
Voltage(V) --        3.00        6.00        6.00        6.00        6.00
TX Bias(mA) --       0.00      131.00      131.00      131.00      131.00
TX Power(dbm) --        neg inf        8.16        8.16        8.16        8.16
RX Power(dbm) --      -11.80        8.16        8.16        8.16        8.16

Flapping the port by disabling it and reenabling it has no effect.
This issue has only started happening since an upgrade to version 4.5(ABBI.1). To fix this we downgraded to 4.4(ABBI.2) but that brings its own issues.

When looking at interface statistics on 4.4(ABBI.2) we get corrupted information;

stack# show interfaces transceiver 1/30
  Transceiver Information

Port : 1/30 (SFP)
Vendor : ��������
Part Number : ��
Serial Number : ��������
Revision : ��
Date Code : 20�-�-�
Transceiver : Others
Calibration : Internal

++ : high alarm, + : high warn, - : low warn, -- : low alarm.
neg inf : negative infinity(0mW).

               Current  High Alarm   High Warn    Low Warn   Low Alarm
                         Threshold   Threshold   Threshold   Threshold
--------------- -------------- ----------- ----------- ----------- -----------
Temperature(C) ++       30.00        0.00        0.00        0.00        0.00
Voltage(V) --        3.00        6.00        6.00        6.00        6.00
TX Bias(mA) --       64.00      131.00      131.00      131.00      131.00
TX Power(dbm) --        2.27        8.16        8.16        8.16        8.16
RX Power(dbm) --      -11.80        8.16        8.16        8.16        8.16

This is happening with a variety of transceiver manufacturers.
«1

All Replies

  • Zyxel小編 Lucious
    Zyxel小編 Lucious Posts: 278  Zyxel Employee
    First Anniversary Friend Collector First Answer First Comment
    Hi @Bash

    We've been recently reported a similar case and now under clarification as well, please help to clarify below info,

    1.
    Did the port dropping (stop transmitting light) issue only happen in firmware 4.5(ABBI.1)?
    Which version of firmware do you confirm with no such issue? 4.40(ABBI.2) or any other?
    2. 
    Given the fact that what we see in this page is all "���", we'd like to confirm if it is some kind of encoding problem between terminal software and browser(HTML). In 4.40(ABBI.2), could you provide a screenshot of corrupted DDMI information on the specific transceiver?
    3.
    For the corrupted info problem, could you try repeating "show interfaces transceiver" command for times to see if any change/difference? Moreover, try unplugging and replugging in the transceiver to see if any improvement.

    Your cooperation will be highly appreciated!
    Zyxel_Lucious
  • Bash
    Bash Posts: 8
    Friend Collector First Comment
    Hi @Zyxel_Lucious

    Thanks again for the quick response. In answer to your questions;

    1.
    Yes, the port dropping only occurred since we upgraded to firmware 4.5(ABBI.1), we since downgraded to  4.40(ABBI.2) and have not seen the issue again. We have not trialled any other version of firmware to look for this issue.

    2. Attached is a screenshot, we see the "���" in the terminal also.

    3. I have tried repeating the command but the result is the same, unfortunately this switch is in a remote location and I cannot unpug and replug any time soon. I will try and replicate the issue on a more local switch.


    Thanks,
    Ben
  • Zyxel小編 Lucious
    Zyxel小編 Lucious Posts: 278  Zyxel Employee
    First Anniversary Friend Collector First Answer First Comment
    edited February 2019
    Hi @Bash

    Regarding the 3rd answer, maybe you can try the command "inactive / no inactive" to bounce the port instead of actual unplugging/replugging.

    Zyxel_Lucious
  • Bash
    Bash Posts: 8
    Friend Collector First Comment
    Hi @Zyxel_Lucious

    As mentioned in my original post 'Flapping the port by disabling it and reenabling it has no effect.' We attempted this first of all but found it took no effect and the only solution was to physcially unplug and replug in the optic.

    Thanks,
    Ben
  • Zyxel小編 Lucious
    Zyxel小編 Lucious Posts: 278  Zyxel Employee
    First Anniversary Friend Collector First Answer First Comment
    edited February 2019
    Hi @Bash

    We're now preparing a debug code (based on 4.50) to analyse the port dropping issue.
    Would it be ok for you to test the code for us? It may also involve some debug command input / log capture.

    We'll PM you for the further info.

    Zyxel_Lucious
  • Bash
    Bash Posts: 8
    Friend Collector First Comment
    Hi @Zyxel_Lucious

    Yes that would be absolutely fine, I can certainly test that for you.

    Thanks,
    Ben
  • Hi @Zyxel_Lucious,

    I would like to follow this thread up. We have deployed a pre release firmware to some of our stacks which mitigates a high CPU bug but regrettably the optic issue still persists and If anything its gotten worse.

    Could you please advise on the best plan of action. We're still having to manually visit site in order to pull and re seat the optic.

    Thanks,

    Matthew

  • Zyxel小編 Lucious
    Zyxel小編 Lucious Posts: 278  Zyxel Employee
    First Anniversary Friend Collector First Answer First Comment
    edited July 2019

    Hi @B4RN_Mercer


    We've been analyzing and co-working with transceiver vendor regarding the SFP link dropping issue for months, it turns out that root cause is seemingly signal instability of the transceivers, depends their hardware design and IC vendor.

    After all, Zyxel transceivers are recommended on the switches because they're well-verified and the interoperability will be guaranteed.


    Sincerely,

    Zyxel_Lucious

  • Hi @Zyxel_Lucious

    We've been collecting some data since the last post to try and correlate the faults to some sort of pattern.

    What we're seeing is that the xgs4600-32f has issues but not the 52f...

    We have approximately 100 32f switches deployed and we're seeing the optic dropping issue almost daily.

    The data we have across multiple different sites is that when the xgs4600-32f is stacked the issues occur on switch 3 pretty much exclusively. There has been times where optics have dropped on other stack members but that is incredibly rare.

    Please can you continue diagnostics as we still believe that there's more to it than just our optic vendor.

    Kindest Regards

    Matthew

  • Zyxel小編 Lucious
    Zyxel小編 Lucious Posts: 278  Zyxel Employee
    First Anniversary Friend Collector First Answer First Comment
    edited June 2020
    @B4RN_Mercer

    Your concern is understandable, but I would like to share that we've had similar experience from other cases and this type of IoP issue usually comes from the quality of different transceiver vendors. 
    Zyxel has fully tested our transceivers to make sure that the system works stable with our own-brand transceivers.

    Local Zyxel sales will contact you shortly to further support your business.


    Zyxel_Lucious