The V3700 is now an old product, and maintenance is not cheap. Units with 4G memory went out of support in 2018, and the rest in 2022.

The hardware in the v3700 processor canister is a fairly standard computer using the usual Intel nic and LSI sas controllers and a dual core Celeron CPU G530T @ 2.00GHz. Memory is either 4 or 8Gb using low profile 4G parity dimms.

It uses either bios boot, or UEFI in later code releases, to boot a linux OS from a SanDisk 32G Sata SSD.

The custom Linux kernel used depends on the code version e.g. code 7.6.1.8 shows a Linux version of 3.10.0-327.36.3.el7_1.sanvc_15C.5.x86_64

The files containing the code, upgrade test and drive firmware are all encrypted with PGP and easy to extract once you have the key.

The secret to maintaining IT system reliability is to look for the early signs of problems before they become a serious issue. The normal logs extracted from the gui contain standard linux logs like dmesg and messages, so are a useful item to check for potential issues.

Downgrade:

Don't expect to retain any data when trying an unsupported firmware or hardware change.

There is no path to downgrade the firmware from 7.6 back to 7.5 via the gui. This would be required to reduce the memory to 4G. It can be done by reimaging the ssd to 7.5, but all data, including storage will be lost.

Changing hardware will also produce an error. Usually it will go after several reboots, provided the cannisters both have the same hardware.

Common Issues

Common issues I have seen include bad blocks on the boot ssd and battery failure. There are smart logs for the ssd, but the battery is only tested at boot.

Rebooting a node periodically would do a basic test of the battery to confirm it is able to properly handle a power loss, where the battery maintains the system long enough to save any cached writes to the ssd.

The battery test logging is found in 'boot/7846125-2.trc' or similar depending on the node id and looks like this (timestamps have been omitted) This battery is 7 years old and has passed a power failure test - definitely not recommended to do on a production system.

partial log of a battery test during boot
If the battery needs charging, the boot process will wait, and if the battery battery fails the test, it will not complete the boot process and come fully online.

ec_tb4s_led_set_led: BBU Status: off ec_bbu_set_control: Battery mode set to CHARGE ec_bbu_init: Battery FRU '00Y4643', serial number 'redacted' c_bbu_init: Battery manufacturing date 2016-01-21, manufacturer name 'CLS ec_bbu_init: Battery h/w v1, controller f/w v12, characterisation v12 ec_bbu_init: Could not find a valid "charge reporting OK" marker file - inferring that the charge reporting may be unreliable ec_bbu_init: Battery manufacturer name 'Palladium' ec_bbu_init: Battery device name 'bq3060' ec_bbu_init: Battery chemistry 'LFP' ec_bbu_status_eeprom_read_and_validate: CALLED ec_bbu_status_eeprom_read_and_validate: Determined missing_POHs as 3274 (0.37 Yrs)

itep_init_EP: The remaining charge reported by the battery has been deemed unreliable - assuming that the battery has insufficient charge to save system data

ec_bbu: BBU status is CHARGE_INSUFFICIENT ec_bbu_poll_main: Battery current 0 mA, voltage 10127 mV, charge 0 mAh (1045 mAh from gas gauge) ec_bbu_poll_main: The battery is not charging. It is possible that this is simply because it is full, so a one-second test test will be performed now in an attempted to stimulate the battery into charging.

ec_bbu_set_control: Battery mode set to DISARMED ec_bbu_set_control: Battery mode set to ARMED ec_bbu_run_battery_test: Battery current 0 mA, voltage 10119 mV, charge 0 mAh (1045 mAh from gas gauge)

ec_bbu_run_battery_test: About to perform a battery test
ec_bbu_set_control: Battery mode set to TEST
ec_bbu_run_battery_test: Battery test PASSED
ec_bbu_run_battery_test: Battery current -2585 mA, voltage 9697 mV
ec_bbu_poll_main: Battery current 1353 mA, voltage 10368 mV, charge 2 mAh (1038 mAh from gas gauge)
ec_bbu_poll_main: Battery current 1117 mA, voltage 10616 mV, charge 4 mAh (1038 mAh from gas gauge)
ec_bbu_get_voltage_mV: i=0 Battery voltage reading of 10353 mV rejected as too dissimilar from previous 10616 mV
ec_bbu_poll_main: Battery current 0 mA, voltage 10353 mV, charge 4 mAh (1047 mAh from gas gauge)
ec_bbu_poll_main: Battery current 0 mA, voltage 10320 mV, charge 4 mAh (1047 mAh from gas gauge)
ec_bbu_poll_main: Battery current 0 mA, voltage 10287 mV, charge 4 mAh (1047 mAh from gas gauge)
ec_bbu_get_status: CHARGE_FULL - the battery is now fully charged
ec_bbu_trace_battery_temperatures: Battery Temperature readings: 19.5C
ec_bbu_poll_main: Battery current 0 mA, voltage 10269 mV, charge 4 mAh
ec_bbu_poll_main: The battery is now full and is reporting a charge of 1047 mAh, and a full charge capacity of 1049 mAh
ec_bbu_trace_battery_temperatures: Battery Temperature readings: 19.5C
ec_bbu_poll_main: The charge reported by the battery is believed to be OK
ec_bbu_set_control: Battery mode set to ARMED
ec_bbu: BBU status is CHARGE_SUFFICIENT_FOR_TWO_FHDS

The partial log from a power failure

++++++++++++++++++++++++++++++++++++++++++++++++++++++
Exiting STATE: ECMGR_PLATFORM_STATE_write_HD4 exit CODE: ECMGR_PLATFORM_MODE_shutdown
++++++++++++++++++++++++++++++++++++++++++++++++++++++
\ ------------------------------------------------------
Entering STATE: ECMGR_PLATFORM_STATE_exit
\ ------------------------------------------------------
itep_stop_EP: CALLED
ECMGR_POWER: ecmgr_power_report_battery_led: CALLED
ec_tb4s_led_set_led: BBU Status: off
itep_stop_EP: INFO - sending AC monitoring abort
ECMGR_PLATFORM_STATE: Normal termination, exit code SS_EXIT_POWERFAIL
ECMGR_PLATFORM_STATE: Normal termination, stimulus 13
++++++++++++++++++++++++++++++++++++++++++++++++++++++
Exiting STATE: ECMGR_PLATFORM_STATE_exit exit CODE: SS_EXIT_POWERFAIL
++++++++++++++++++++++++++++++++++++++++++++++++++++++
ec_tb4s_led_set_led: STATUS: off
ec_tb4s_led_set_led: IDENT: off
ec_tb4s_led_set_led: FAULT: off
fcoe fast_node_reset - Reset all pports!
fcoe fnr reset total time 0 ms
sas fast_node_reset - resetting ports sas fast_node_reset - ports reset
ECMAIN exiting with CODE: SS_EXIT_POWERFAIL (internal is SS_EXIT_POWERFAIL)
ecmain exit with 13, SS_EXIT_POWERFAIL
validate passed
switching off the battery
power failed shutting down

The ssd filesystem changes occasionally for different code releases.

#this is version1 (for 7.5) partitioning
Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 1 787184 787184 384.4M 83 Linux
/dev/sdc2 787185 4080509 3293325 1.6G 83 Linux
/dev/sdc4 4080510 62524979 58444470 27.9G f W95 Ext'd (LBA)
/dev/sdc5 4080511 20675654 16595144 7.9G 83 Linux
/dev/sdc6 20675656 21205799 530144 258.9M 83 Linux
/dev/sdc7 21205801 50058539 28852739 13.8G 83 Linux
/dev/sdc8 50058541 51118829 1060289 517.7M 83 Linux
/dev/sdc9 51118831 54813779 3694949 1.8G 83 Linux
/dev/sdc10 54813781 61994834 7181054 3.4G 83 Linux
/dev/sdc11 61994836 62524979 530144 258.9M 83 Linux

#This is 7.6 partitioning with UEFI
fdisk -l /dev/sdc
Disk /dev/sdc: 29.8 GiB, 32017047552 bytes, 62533296 sectors
Disk model: SanDisk SD6SA1M0
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00015fb2

Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 2048 208844 206797 101M ef EFI (FAT-12/16/32)
/dev/sdc2 208845 1140614 931770 455M 83 Linux
/dev/sdc3 1140615 4080509 2939895 1.4G 83 Linux
/dev/sdc4 4080510 62524979 58444470 27.9G f W95 Ext'd (LBA)
/dev/sdc5 4080511 20675654 16595144 7.9G 83 Linux
/dev/sdc6 20675656 21205799 530144 258.9M 83 Linux
/dev/sdc7 21205801 50058539 28852739 13.8G 83 Linux
/dev/sdc8 50058541 51118829 1060289 517.7M 83 Linux
/dev/sdc9 51118831 54813779 3694949 1.8G 83 Linux
/dev/sdc10 54813781 61994834 7181054 3.4G 83 Linux
/dev/sdc11 61994836 62524979 530144 258.9M 83 Linux