AI Hardware — Rack, Interconnect and Identification
The physical layer of an AI datacenter, taught the way the work is actually done: identify the thing in order to install it, verify it, or replace it. Ordered facility-inward — safety, then the rack, then the node, then what goes in it, then the cabling that is the largest part of the job, then testing and bring-up. Anchored on NVIDIA's NCP-ARI blueprint, whose audience is 'Data Center Infrastructure Technicians' and for which NVIDIA has published no training.
8 of 19 units built
Units A1–A8 are WRITTEN — safety, the rack frame, reading a server from outside, inside the node, GPU form factors, verifying what you were shipped, site assessment and rack preparation. A9 onward is syllabus: the shape is visible and deliberately carries no links, because every source is fetched and its content checked before it ships. The shape was rebalanced in s183 against published certification weightings rather than against what was easiest to find material for: cabling and its verification are the heaviest part of the path, as they are in every professional blueprint, and identification survives as a differentiator bound to a task rather than as the spine. Safety is first, deliberately against how most facility curricula sequence it. 🔴 Stated plainly: the retrieval practice for these units does not exist. The two AI-hardware decks were written from the procurement and architecture side, so A1, A2 and A8 have no drill at all, and the rest link to a partial one that says on each unit exactly what it does and does not cover. That drill is the outstanding debt on this path — it has grown with every unit written.
Track A — The physical layer, facility inward
Nineteen units from the safety brief to timed identification under real conditions. Weighted to match what certification bodies actually weight — roughly half the path is cabling, its support systems and its verification — with two units that are genuinely ours, because no curriculum anywhere teaches datacenter hardware identification as a discipline.
A1
Safety, ESD and handling — before you touch anything
KR ~100m
🔴 First, deliberately, and against how most facility curricula sequence it. ESD damage leaves no mark and fails weeks later, a populated GPU chassis is a two-person lift, an unterminated transceiver emits invisible laser radiation, and busbars are live. ANSI/ESD S20.20 lists training as program element #1 and NFPA 70E demands demonstrated supervised performance rather than a certificate. Almost entirely Risk: identify the hazard AND its mitigation, which is not what a definition card tests.
After this you can
Name the four hazard classes around AI hardware — electrical, electrostatic, mechanical, optical — and the specific mitigation each one takes
Explain latent ESD failure, and say why it changes how you troubleshoot a machine that passed its acceptance test
State what an ESD control programme requires of a person, not just of a bench
Decide before opening a rack whether the job needs a second person, a lift, or a de-energised chassis
Do these in order
1 ReadSafety NVIDIA DGX H100/H200 User Guide · ~25m
Read all of it — it is short, free, and it is the manufacturer's own brief for exactly the class of machine this path is about. Concentrate on Equipment Handling Practices, Electrical Precautions and Rack Mount Warnings. The weight and two-person language is not legal boilerplate: a populated GPU chassis is genuinely past a safe single-person lift.
Read for the mechanism, not the ritual: triboelectric charging, the human body model, and above all LATENT failure — a part that is damaged, still works, and dies later. That one concept is why the whole discipline exists. You can skip the device-physics detail on failure modes at the junction level.
Now the standard itself, in summary. Read it for one question: what does S20.20 require of PEOPLE? Training is program element #1, ahead of any mat, strap or bench — which is the argument for why this unit is first in the path rather than last.
Read the overview and the hazards section. You are not being asked to become an electrician — you are being asked to recognise the situations where a rack has stored or alternate energy sources and a technician's ordinary habits are not enough.
Skim for the hazard taxonomy and the arc-flash material. This is the free, public-domain half of what NFPA 70E covers; 70E itself sits behind a free NFPA account if you want the normative text, and its central idea is that competence must be demonstrated under supervision, not attended in a classroom.
What to watch for
The instinct is to treat this as the compliance chapter you skim on the way to the hardware. Two things earn the time. First, ESD is the only hazard here with no feedback loop — you will never feel the discharge that wounds a DIMM, the machine will POST perfectly, and it will fail six weeks later looking exactly like a software problem. Strap on before the chassis is open, every time, is a habit you build now or build after an expensive week. Second, the documents will tell you a rack is heavy; they will not tell you the actual failure mode, which is a rack tipping while a chassis is extended on its rails. Stabilisers down, one node out at a time. Everything after this unit assumes you already do both without being told.
A2
The unit of account: node, rack, row, hall
K ~115m
You cannot place a part until you know what it plugs into. Rack units and why a GPU server is 4U or 8U rather than 1U; the rack as a power and thermal budget rather than a shelf; OpenU versus EIA 19-inch and why the two pitches are not interchangeable; hot and cold aisles; the row/pod/hall hierarchy that decides where a cable can physically reach. NCP-ARI weights this frame at 20%.
After this you can
Convert between U, millimetres and inches without looking it up, and say why an 8U GPU chassis is 8U
State the two rack pitches — 44.45 mm EIA and 48 mm OpenU — and explain why gear built for one does not mount in the other
Describe a rack as a power and cooling budget, and say what runs out first in an AI row
Read a rack elevation or a datacenter photograph and name what you are looking at: aisle orientation, containment, busbar, manifold, power shelf
The spine of this unit, and the only openly-licensed textbook in this whole path — Google's own, free, CC BY 4.0, figures included. Read for the building-outward view: power distribution, cooling topology, and where a rack sits inside it. Fig 5.23 (an end-of-row CDU with its headers, hoses and quick-disconnects) and Fig 5.22 (copper cold plates) are the two to sit with. You are reading for orientation, not for facility engineering — do not get lost in the efficiency mathematics.
Do NOT read this cover to cover — it is a manufacturing specification and you are not building a rack. Go to §6.1.2 and read the one sentence that defines 48 mm OpenU alongside 44.45 mm EIA 310-D, then look at the drawings of both pitches to the same scale. Then find the 48 V busbar cross-section in §6.3. Those two figures are the whole point: the reason a rack full of OCP gear and a rack full of 19-inch gear are not the same object.
The landing page for the whole Open Rack family — frame, power shelf, BBU, blind-mate manifold. Bookmark it rather than read it. Deep PDF links on this site change filename between revisions; this page does not.
The current top end, and the reference point for how much a rack can now be a single machine. Two figures do the work: four 72-GPU racks side by side, and the exploded rack rear view — which labels the power busbar, the liquid cooling manifolds and hoses, the cable cartridges, the bezel mounting braces and the seismic bracing. Compare that rear view against anything in Chapter 5 and you can see what changed and what did not.
Watch for why containment exists at all — that a hall's cooling capacity is destroyed by air that goes round the equipment instead of through it. Aisle orientation is the first thing you will be asked to get right when you place a rack, and it is the easiest thing to get backwards.
A real deployed cluster rather than a render. Watch it for scale and for the row-level view — how many racks make a pod, what the aisles actually look like when they are full, and where the cabling runs. Treat the vendor framing as vendor framing; you are here for the room, not the pitch.
Not a document — 100+ captioned photographs across twelve sites, and the best free hardware photography anywhere. Spend the time deliberately: for each image, name what you are looking at BEFORE reading the caption, then check yourself. That is the drill this unit does not otherwise have.
What to watch for
The trap in this unit is treating a rack as furniture — a shelf you put servers on until it is full. It is not. It is a power budget, a thermal budget and a weight budget, and in an AI row you will hit one of those long before you run out of U. A 42U rack with 8U GPU chassis in it does not hold five of them; it holds however many the feed, the cooling and the floor will carry, which is often two or three. The second trap is assuming 19-inch is universal because it has been for thirty years. OpenU is 48 mm against EIA's 44.45 mm, the two look nearly identical in a photograph, and gear built for one does not mount in the other.
A3
Reading a server from the outside — and what its indicators are telling you
KR ~110m
The first real-world act is at the front of a rack with the doors shut. Chassis height and bezel, drive-bay layout, intake shape, and the OCP-versus-standard-EIA distinction that looks identical in a photograph and is not interchangeable. 🔴 Then the half the first draft missed entirely: service tag and part-number location, model nomenclature decoding, system ID and fault LEDs, PSU and drive status, blink codes and the POST display. Three separate Dell blueprints weight this — 'Explain the hardware fault indicators', 'Analyze the visual indicators on server components — system ID, PSU, and BLINK', 'Perform visual inspections in cases where automated monitoring is unavailable'. Misreading an amber LED has consequences, so it carries Risk.
After this you can
Identify a server's height, bay layout and front/rear I/O from a photograph, and say what configuration that implies
Read a front and rear panel: locate the ID button, the power and fault indicator, PSU status and drive activity/status
State what solid green, fast-blink green, amber and blinking amber each mean, and what you do next in each case
Find the identity of a machine you have never seen — service tag, part number, model nomenclature — without powering it on
Use a POST or diagnostic code as evidence rather than as an error message
The single best free document for this unit: a current 8-GPU AI server documented end to end, with callout-numbered front and rear views and — the part that matters — full Front Panel and Rear Panel LED Indicator tables giving every colour and blink state against its meaning. Read down to the end of 'Rear Panel LED Indicators' and stop there; the component-location sections after it belong to A4. Do not memorise Cisco's specific states. Learn the SHAPE: every vendor publishes this same table, and knowing it exists is most of the skill.
A second vendor doing the same job differently, which is the point of reading two. Focus on 'Front Panel Connections and Controls', 'Rear Panel Modules' and 'BMC Port LEDs'. Note the bezel: with it on, this machine is almost unidentifiable, and that is the normal state you will find it in.
Short and directly on the objective three Dell exam blueprints weight. Watch for the distinction between the system ID indicator — which you turned on yourself to find the box — and a fault indicator, which the machine turned on to tell you something. Confusing the two is the classic first-week mistake.
PSU indicators specifically, because they are the ones you read most often and the ones with the most ambiguous states. Watch for what an amber PSU LED does and does not tell you: it can mean the supply, the feed, or the fact that somebody pulled the wrong whip.
5 ReadPOST Error Codes Cisco UCS C885A M8 Installation and Service Guide · ~20m
Do not read this list — nobody reads this list. Open it, scroll it, and take away two things: that a machine which will not boot is usually telling you exactly where it stopped, and roughly what stage of the boot the codes are describing. The skill is knowing this reference exists and how to look a code up under pressure, not remembering any of it.
What to watch for
Amber does not mean broken. On most platforms it means 'something here needs a human', which covers a genuinely failed part, a part that is fine but seated wrong, a redundancy you are no longer getting, and a maintenance flag nobody cleared. Acting on the colour alone is how a healthy PSU gets swapped. The indicator narrows where to look; the log, the BMC or the POST code tells you what actually happened. The second trap is the bezel — a locked front bezel hides most of what this unit teaches you to read, and the identity you need is often on a pull-out tag or a rear label rather than the front at all.
⚠️ Partial, and named honestly: this deck is about choosing and sizing accelerators, and NOT ONE of its 38 cards tests a chassis indicator, a service tag, model nomenclature or a POST code. It is here for adjacent context — DGX versus HGX, the XE9680 — not for retrieval practice on this unit. The drill for A3 does not exist yet.
A4
Inside the node — mainboard, sockets, DIMMs, risers, backplanes
K ~120m
Layout reveals design intent. Socket count, memory channel population, which PCIe lanes come off which socket, direct versus backplane drive bays. A GPU on the wrong riser is lane-starved and the symptom is 'it works but it is slow'; 'the second CPU is missing so half the slots are dead' is a real support call. Read the board and you can predict both before powering anything on.
After this you can
Open a chassis, or a top-down photograph of one, and name the major assemblies: CPU tray, DIMM banks, risers, mid-plane, power distribution board, fan wall, management board
Say which slots are fed by which socket, and predict what dies when a second CPU is absent
Explain why DIMM population order is a performance decision and not a tidiness one
Distinguish a direct-attached drive bay from a backplane, and say what each implies about what the chassis will accept
Read Fig 6.3 before you read the prose. It is an annotated top-down photograph of an open two-socket tray with leader lines to the fan wall, DIMM banks, heat-pipe heatsinks, PCIe risers and drive bays — one image that carries most of this unit. Then Fig 6.4, which puts Intel, AMD and Arm trays side by side so you can see what varies and what does not. Free, CC BY 4.0, and the only source in this path you could legally reproduce.
The same document you read in A3, now for the second half you deliberately skipped. Start at 'Component Locations' and work through Mainboard, DC-SCM, Power Distribution Board, both Mid-Plane boards, the riser boards, and DIMM Slot Locations. Read it as an inventory of what a modern AI node is actually made of — note how many separate boards there are, which is the thing a two-socket general-purpose server does not prepare you for.
Jump to 'Motherboard Connections and Controls', 'Motherboard Tray Components' and 'GPU Tray Components'. The split into two trays is the design decision worth noticing: the compute and the accelerators are physically separate assemblies, serviced separately, and that shapes everything about how the machine is worked on.
A real teardown, which no document gives you: the order things come apart in, how much cabling is actually in there, and what the airflow path looks like when you can see it. Watch for where the risers sit relative to the CPUs — that geometry is the lane-starvation story this unit is about.
Reference, not reading. Use it to fix the module families in your head — registered versus multiplexed-rank, and where CXL memory modules sit alongside them — so that a photograph of a populated DIMM bank stops being a row of identical green sticks. Ignore the part numbers and the sales copy.
What to watch for
The wrong mental model here is that a slot is a slot. It is not: on a two-socket board the PCIe lanes are divided between the sockets, so a card in the wrong slot is either lane-starved or reaching across the interconnect to get to memory, and in both cases the machine works — just slower, intermittently, and in a way no error message will ever mention. The same is true of DIMMs, where an unbalanced population quietly costs you memory bandwidth the machine will never complain about. This is the unit that teaches you to predict a performance problem from a photograph, which is a genuinely different skill from finding a broken part.
⚠️ Partial. Three cards land on this unit's consequences — the NUMA penalty, why the host CPU is not an afterthought, and the PCIe Gen5 numbers — but the deck was written from the architecture side and nothing in it tests board layout, DIMM population or riser topology. Useful after this unit, not sufficient for it.
A5
GPU form factors — PCIe, SXM and OAM, and what each one lets you service
K ~135m
🔴 The most consequential physical distinction in the path, and it is taught here bound to its consequence rather than as recognition for its own sake. The three differ in power delivery, cooling, whether NVLink exists at all, and — the part that decides your day — whether the thing is field-replaceable. ⚠️ Worth stating plainly to the learner: no vendor publishes a GPU module removal procedure, because SXM and OAM modules are not customer-replaceable. That absence is itself the lesson about field-replaceability, and it is why this unit teaches the module from specification drawings.
After this you can
Tell PCIe, SXM and OAM apart from a photograph or a front-view drawing, and say what each implies about power, cooling and interconnect
State what is field-replaceable in each case, and what a failed accelerator means for the machine it is in
Explain what a baseboard is, and why an 8-GPU node is one assembly rather than eight parts
Read an accelerator's physical spec — module size, mezzanine connector, power envelope — from the OAM specification rather than from marketing
Start here, at the index. One product line, three published front views — 4-DW PCIe, 8-DW PCIe and SXM5 — each with its own callout figure. Nothing else found anywhere teaches this distinction as cleanly, precisely because it holds the vendor, the chassis and the generation constant and varies only the thing you are learning.
The PCIe case. Read the callouts and note what the GPUs are: ordinary double-wide add-in cards in numbered slots, individually removable, each drawing its own power. Then open the 8-DW page (`/sr675-v3/server_front_view_8dw_model`) and see the same idea at twice the density. Ask yourself what has to give at eight cards — that question is the reason SXM exists.
The same chassis, and now a different machine. Four labelled front-view figures cover the SXM5 PCIe switch board, the CX-7 mezzanine board with its OSFP port card, the GPU-L2A assembly and the interposer card, in both 2.5-inch and E3.S drive configurations. 🔴 Read the callouts and notice what is NOT in them: there is no line item for an individual GPU. That is the whole unit in one absence.
The open answer to SXM, and the only place you can see this class of module dimensioned rather than photographed. Go to the dimensioned bottom view early on — the 102 × 165 mm module with its mezzanine connectors underneath — and the figure showing four adjacent modules with heatsinks fitted. Read the mechanical and thermal sections and skip the electrical signalling entirely. ⚠️ Sole connector interface is the Molex Mirror Mezz; you will see it miscredited elsewhere.
Ten minutes for the hierarchy that the specification assumes you already have: module, baseboard, tray, chassis. Get that nesting straight and the phrase 'an 8-GPU baseboard' stops being jargon and starts being a physical object you can picture.
The SXM baseboard as a product. Watch for how the eight GPUs are presented — as one assembly with one thermal solution and one interconnect fabric, not as eight cards that happen to be adjacent.
The same class of hardware taken apart by someone with no product to sell. Watch specifically for how the baseboard is handled and what has to come out before it moves — that sequence is the practical meaning of 'not field-replaceable'.
One chassis documented against four different accelerator architectures — NVIDIA A100 and H100/H200, AMD MI300X and Intel Gaudi3 — with the internals shown side by side. Nowhere else publishes that comparison for free. Look at the figures, read the physical and power sections, and skip the configuration and management chapters.
What to watch for
The framing to avoid is 'SXM is the fast one'. The distinction that survives contact with the job is what each form factor lets you SERVICE. A PCIe accelerator is a card: it fails, you pull it, you fit another, the rest of the machine is unaffected. An SXM or OAM module is soldered to a baseboard carrying all eight, and no vendor on earth publishes a removal procedure for one — because there isn't one. A single failed module means the baseboard is the replaceable unit, which is a five-figure part and a very different conversation with whoever owns the machine. That absence of documentation is not a gap in this course; it is the fact being taught.
⚠️ Partial, and the gap is specific. Two cards are directly relevant — 'SXM vs PCIe form factor — what's the trade?' and 'DGX vs HGX' — but both answer from the procurement side (bandwidth, NVLink, power), not from the serviceability side this unit is built on. 🔴 OAM does not appear anywhere in the deck's 38 cards, and neither does the baseboard-as-replaceable-unit consequence. Both are unwritten.
A6
Verifying the accelerator you were actually shipped
KS ~105m
Identification bound to a purpose: confirming that what arrived is what was ordered, and that what is installed is what the inventory claims. Board length, slot width, power-connector shape and count, shroud and heatsink, memory package layout, and where the real part numbers are printed. ⚠️ This unit and A19 are the path's genuine differentiators — no curriculum anywhere teaches datacenter hardware identification as a discipline. They are deliberately sized as a differentiator rather than as the spine, and each is tied to a task a technician actually performs.
After this you can
Tell five current PCIe accelerators apart on sight by length, slot width, shroud, heatsink and power connector
Find the authoritative identity of a card — the silkscreen and the printed part number — and say why the box, the label and the invoice are all weaker evidence
Check an installed accelerator against a bill of materials without powering the machine on
State what is field-replaceable in a given chassis by reading its service documentation rather than assuming
🔴 The reason this unit can exist: five current PCIe accelerators — H200 NVL, H100 NVL, RTX PRO 6000, L40S and L4 — photographed individually in one free document. Go to the GPU section and study the photographs side by side before reading a word of the text. Ask of each: how long, how many slots wide, what does the shroud look like, how many power connectors and of what shape. That comparison is not published anywhere else for free.
The service view, which is where identification stops being a quiz and starts being a job. Read 'Service Considerations' and 'Alignment Features', then look at what the replacement procedures actually cover: the CPU tray, fan modules, PSUs, drives, the DC-SCM, the BlueField-3 DPUs — and the HGBB tray as a single unit. 🔴 That last one is the A5 lesson made concrete: the baseboard tray is the replaceable part, and there is no procedure below it.
A PCIe accelerator handled rather than rendered. Watch for the physical details a datasheet never gives you — how it is retained, what the airflow assumption is, and how it looks once it is in a chassis surrounded by seven others.
A Short, and included deliberately for that reason: it is roughly the amount of time you get to look at a card when someone hands it to you. Name it before the narration does.
High-resolution silicon and board photography, used here for one skill: reading a silkscreen. ⚠️ Worth knowing that a file in this gallery labelled as one part carries the silkscreen of an older, different one. That is not a trap set for you — it is the ordinary condition of vendor asset libraries, and it is exactly why this unit insists the marking on the board outranks the filename, the label and the packing list.
What to watch for
The failure this unit prevents is quiet and expensive: the machine works, so nobody checks, and the inventory says something the rack does not contain. It happens because every convenient identifier is second-hand — the carton, the asset label, the line on the purchase order, the filename on a vendor image. Only two things are first-hand, the silkscreen on the board and the part number printed on it, and both take a torch and a minute. Build the habit of going to those, in that order, before you believe anything else. ⚠️ And note the asymmetry with A5: a PCIe card can be verified by looking at it, while an SXM or OAM module usually cannot be seen at all without disassembling an assembly you are not meant to disassemble. For those, the verification is documentary — the baseboard's own label, and what the management interface reports.
The closest fit in the deck, and genuinely useful here — cards on H100 vs H200, L40S and L4, the RTX PRO 6000 and the XE9680 name the same parts this unit teaches you to recognise. ⚠️ But they test what a part is FOR, never how to tell it apart from the one next to it. The recognition half is still unwritten.
A7
Pre-deployment planning and site assessment
KR ~125m
Everything that must be true before hardware arrives, and the survey that establishes it: floor load capacity against a populated rack's weight, clearances for doors and lifting gear, delivery path and lift access, power availability and phase balance, environmental readiness, and the inventory check against the bill of materials. NCP-ARI weights this 11% and BICSI's design exam gives concept planning 30% — the first draft had nothing for it.
After this you can
Run a site survey against a specific delivery: floor loading, ingress path, door and aisle clearances, lift access
Work out how many AI systems a rack can take, and say which limit binds first — power, heat, weight or U
Read a facility's power feed and say whether it supports the intended density, including what phase balance means for it
Name what must be signed off before hardware ships, and what it costs to discover each item late
🔴 The spine of this unit, from the vendor whose exam this path is anchored on. Work through Coordination, The Economy of Data Center Resources, Density of Compute Racks, Safe System Delivery, Power and Heat Dissipation, and Environmental Thermal Guidelines. Sit with the density section in particular: a DGX H100 is 10.2 kW in 8U, and NVIDIA recommends four per rack and no more — not because a 42U rack lacks the space, but because the thermodynamics stop working. That single number is the clearest statement in this path of why a rack is a budget rather than a shelf.
Read this for the PLANNING half only — Data Center Power Configuration, Power Redundancy and what N+1 actually buys, and Phase Balancing. The connector and whip taxonomy is A9's job; leave it. What you want here is the ability to look at an available feed and say whether the density somebody has asked for is possible at all.
A second and very different voice: a vendor stating exactly what a customer site must provide before anything ships. Read Facility and Power, and note how concrete it gets — named inlet types (L6-30P, IEC 60309, CS8365C), specific clearances, specific fibre and connector requirements. This is what a real pre-deployment checklist looks like when a company has been burned enough times to write one down.
The whole electrical path in one pass, upstream of anything you will touch. You are not being asked to design it. You are being asked to know what sits between the grid and the whip you plug in, so that 'is there power available' becomes a question you can actually ask someone.
An operations manager walking the spaces — single cabinets, cages, suites. Watch it for the vocabulary you will be given in a work order, and for the physical constraints of a shared facility, which are stricter and less negotiable than in a room your employer owns outright.
What to watch for
Everything in this unit is cheap now and ruinous later, and that asymmetry is the whole point. A floor that will not take the load, a doorway the crate does not fit through, a feed that supports three systems when four were ordered — each is an afternoon's checking beforehand and a cancelled installation with a truck outside afterwards. The specific trap for someone coming from IT is thinking in U. Space is almost never what runs out first in an AI row; power runs out, then heat, then floor loading, and you can have twenty free U in a rack you must not put anything else into. ⚠️ Second trap: the delivery path is part of the site. A machine that fits the rack perfectly and cannot get to the room is a failed deployment, and nothing about the rack specification will warn you.
⚠️ Thin, and adjacent rather than aligned. A handful of cards touch the same physics from the architecture side — power draw per GPU, why liquid cooling stopped being optional, what is gating buildouts — but nothing in either deck tests floor loading, ingress, clearances or a survey. Treat it as background reading, not as the drill.
A8
Rack infrastructure preparation
KS ~115m
Preparing the rack to receive equipment: rails and cage nuts, mounting depth, blanking panels and brush strips, side panels, containment fit, grounding and bonding, and levelling and seismic bracing. Unglamorous, load-bearing, and the reason a later install goes smoothly or does not. NCP-ARI 10%.
After this you can
Prepare a cabinet to receive a heavy chassis: mounting depth, rail fit, cage nuts, levelling, stabilisers, bonding
Say when a job needs a server lift rather than people, and what point load means for the floor under it
Spot the airflow faults that a photograph of a rack will show you — missing blanking panels, unsealed U-slots, reversed-airflow equipment, broken containment
Recognise an OCP rack on sight by its frame features, and say what it will and will not accept
🔴 This one page is most of the unit. Read Rack Standards and Requirements, Options When Ordering Cabinets, Cabinet Mounting, Seismic Considerations, Cabinet Selection vs. Cable Lengths, Server Mounting Requirements, Racking Servers, Static Weight and Point Load, and Server Lifts. The point-load section is the one to slow down on: it is not the rack's total weight that fails a floor, it is the load concentrated under four castors or feet.
The same job from the equipment side. Read 'Installation Warnings and Guidelines', 'Rack Requirements', 'Rack Specification', 'Slide Rail Specifications' and 'Tools Required for Rack Installation'. Note that the rail specification constrains the RACK, not just the server — post-to-post depth and hole pattern are requirements a cabinet either meets or does not, and finding out at install time is too late.
🔴 Sixteen labelled figures showing the difference between a rack prepared properly and one that is not: equipment securely fixed, open cabinet with poor air containment, cabinet infill panels, equipment airflow direction, incorrect versus correct air containment side by side, cables through a U-slot with brush or foam, side panels, adequate cable management, raised floor sealing. Read the Airflow Management Policy in full. ⚠️ Note that these are enforced contractual rules with a remediation process attached, not advice — which tells you how much operators believe this matters. (Append `.md` to the URL for a clean plain-text version.)
How to know an OCP rack when you are standing in front of one. Look for the named frame features — OpenU pattern retention holes, equipment stop lances, cable lashing lances — and connect them back to the 48 mm pitch you met in A2. These are the visible consequences of that standard, and they are what tells you at a glance which of your gear will mount here.
A cabinet as a manufactured product rather than an abstraction — frame, mounting angles, panels, the adjustments that exist and the ones that do not. Short, and it makes the written specifications above concrete.
What to watch for
This is the unit people skip, and it is the one that decides whether the next three days go well. Two specifics worth carrying. First, blanking panels are not tidiness — an unblanked U is a hole that lets hot exhaust return to the intake, and a rack with a dozen of them can be pulling in its own waste heat while the room's cooling reads as adequate. That is why operators write it into contracts. Second, a rack that is level, stabilised and bonded is invisible when it is right and dangerous when it is not, and both states look identical from the aisle. ⚠️ The mistake with the worst consequences is treating a server lift as optional because two strong people are available. Point load, awkward extension on rails and a 130 kg chassis at chest height is how someone gets hurt, and it is the exact scenario A1 asked you to plan for rather than improvise.
A9
Power delivery — PSUs, whips, connectors, busbars and PDUs
KRplanned
Redundant hot-swap supplies and what N+1 actually promises; EPS versus PCIe versus 12V-2x6 by shape; busbar-fed chassis with no visible power cables; rack PDU and whip plug taxonomy — NEMA L5-30R and L6-30R, IEC 60309, IEC 60320 C13/C19; single versus three phase and what unbalanced phases look like on a meter. ⚠️ Genuine hazard content: a partially seated high-current connector is a documented fire mode.
A10
Cooling hardware — air, direct-to-chip, and the facility side
KRplanned
Airflow direction and the reversed-airflow switch that is a fault rather than a feature; passive versus active heatsinks; then cold plates, quick disconnects, in-rack manifolds, CDUs and rear-door heat exchangers. Risk is real: coolant near energised electronics, and hot-aisle temperature as a personnel hazard with published OSHA limits.
A11
Storage — form factors and what each one signals
Kplanned
U.2, U.3, E1.S, E1.L, E3.S and M.2 boot, plus legacy SAS and SATA, and how a backplane reveals what a chassis will accept. The connectors are deliberately similar and compatibility is not symmetric — a U.3 backplane takes a U.2 drive, not the reverse, and that asymmetry reduces to a single pin in the specification. Obvious in a datasheet, invisible in a photograph.
A12
Network adapters and DPUs
Kplanned
The card that separates a general-purpose server from a member of an AI fabric. Port count, cage type, single versus dual port; and NIC versus DPU, which are physically similar and do completely different jobs — a DPU has its own CPU complex, memory and storage. In an HGX node the split is typically eight ConnectX HCAs plus one BlueField DPU, and that is the thing most often got wrong.
A13
Cables and optics — the heaviest unit in the path
KSplanned
🔴 30% of NCP-ARI, the single largest domain in any blueprint found, and the identification practitioners get wrong most. DAC versus AOC versus pluggable transceivers all terminate in the same-looking cage and decide reach, cost, power and whether the link comes up at all. Form-factor generations SFP through QSFP-DD and OSFP; MPO versus LC; breakout cables; polarity. Assessment is sorting a real tray of cables against a requirement, under time.
A14
Cable support systems and weight management
KSplanned
Physically distinct from A13 and separately weighted by NCP-ARI at 7%: ladder rack and trapeze, overhead versus underfloor pathways, fill ratios, bend radius, slack management and service loops, containment clearance, and the fact that a fully populated cable tray is a structural load somebody had to calculate. Includes the 50/50 left-right routing strategy NCP-ARI names explicitly.
A15
Fabrics — which cable belongs to which network
Kplanned
Four networks share a rack and look alike: scale-up interconnect inside the chassis, scale-out between nodes, the storage network, and out-of-band management — the copper port nobody photographs and everybody needs at 3am. Once you can name the fabric a given cable serves, a rack photograph stops being noise and becomes a topology diagram.
A16
Testing, verification and documentation
KSplanned
🔴 12% of NCP-ARI and absent from the first draft entirely. Continuity and link testing, optical power measurement, bit-error-rate testing, fibre end-face cleaning and inspection — the step whose omission causes a large share of link failures — plus labelling standards, as-built diagrams, and photographing the install. NVIDIA treats this as a discipline; so should we. Ends in a produced artifact, so no quiz can close it.
A17
Bring-up and commissioning
KSplanned
Turning installed hardware into a working node: out-of-band and BMC access, firmware levels and update order, TPM and security posture, POST and hardware validation, and the minimum-to-POST bisection when it does not. NCP-AII weights system and server bring-up at 31% and cluster test and verification at 33% — between them the majority of that exam.
A18
Reference systems and how to recognise one
Kplanned
Integrated system, baseboard sold to partners, and modular architecture are three different things marketed under similar names. Knowing which one you are looking at tells you who owns the design, what is field-serviceable and what the support path is — recognition the professional-level material assumes you already have.
A19
Timed identification under real conditions
Splanned
The capstone, and the second of the path's two differentiators. Real photographs, poor lighting, partial views, obscured labels, against the clock — because that is the condition the job is done in. A pass means naming the part, its form factor, its role, and what you would check to confirm. ⚠️ Deliberately assessed as a task with a decision at the end rather than as a recognition quiz, because the evidence says identification transfers when it is bound to a purpose and may not when it is not. No quiz closes this unit.
The operations side of this hardware — cert prep, decks and GPU labs — lives on the AI & GPU Infrastructure subject hub. This page is the course; that page is everything else worth having open.