tag:blogger.com,1999:blog-82000981029090411782024-03-18T13:31:08.406-04:00Shane ColtonA collection of my personal engineering projects including small electric vehicles, motor controllers, robots, flying things, and other fun electromechanical stuff!Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.comBlogger238125tag:blogger.com,1999:blog-8200098102909041178.post-79951974934658628042024-01-22T00:00:00.001-05:002024-01-22T09:14:43.980-05:00PCIe Deep Dive, Part 4: LTSSM<p>The Link Training and Status State Machine (LTSSM) is a logic block that sits in the MAC layer of the <a href="https://scolton.blogspot.com/2023/06/pcie-deep-dive-part-2-stack-and.html">PCIe stack</a>. It configures the PHY and establishes the PCIe link by negotiating link width, speed, and equalization settings with the link partner. This is done primarily by exchanging Ordered Sets, easy-to-identify fixed-length packets of link configuration information transmitted on all lanes in parallel. The LTSSM must complete successfully before any real data can be exchanged over the PCIe link.<br /></p><p>Although somewhat complex, the LTSSM is a normal logic state machine. The controller executes a specific set of actions based on the current state and its role as either a downstream-facing port (host/root complex) or upstream-facing port (device/endpoint). These actions might include:</p><p></p><ul style="text-align: left;"><li>Detecting the presence of receiver termination on its link partner.</li><li>Transmitting Ordered Sets with specific link configuration information.</li><li>Receiving Ordered Sets from its link partner.</li><li>Comparing the information in received Ordered Sets to transmitted Ordered Sets.</li><li>Counting Ordered Sets transmitted and/or received that meet specific requirements.</li><li>Tracking how much time has elapsed in the state (for timeouts).</li><li>Reading or writing bits in PCIe Configuration Space registers, for software interaction.</li></ul><div>Each state also has conditions that trigger transitions to other states. All this is typically implemented in gate-level logic (HDL), not software, although there may be software hooks that can trigger state transitions manually. The top-level LTSSM diagram looks like this:</div><div><br /></div><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_000.svg" width="640" /></div><div><br /></div><div>The entry point after a reset is the Detect state and the normal progression is through Detect, Polling, and Configuration, to the golden state of L0, the Link Up state where application data can be exchanged. This happens first at Gen1 speed (2.5GT/s). If both link partners support a higher speed, they can enter the Recovery state, change speeds, then return to L0 at the higher speed.</div><div><br /></div><div>Each of these top-level states has a number of substates that define actions and conditions for transitioning between substates or moving to the next top-level state. The following sections detail the substates in the normal path from Detect to L0, including a speed change through Recovery. Not covered are side paths such as low-power states (L0s, L1, L2), since just the main path is complex enough for one post.</div><div><br /></div><h3 style="text-align: left;">Detect</h3><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_001.svg" width="640" /></div><div><br /></div><div>The Detect state is the only one that doesn't involve sending or receiving Ordered Sets. Its purpose is to periodically look for receiver termination, indicating the presence of a link partner. This is done with an implementation-specific analog mechanism built into the PHY.</div><h4 style="text-align: left;">Detect.Quiet</h4><div>This is the entry point of the LTSSM after a reset and the reset point after many timeout or fault conditions. Software can also force the LTSSM back into this state to retrain the link. The transmitter is set to electrical idle. In <a href="https://docs.xilinx.com/r/en-US/pg239-pcie-phy">PG239</a>, this is done by setting the <span style="color: #01ffff; font-family: courier;"><b>phy_txelecidle</b></span> bit for each lane. The LTSSM stays in this state until 12ms have elapsed or the receiver detects that any lane has exited electrical idle (<b><span style="color: #01ffff; font-family: courier;">phy_rxelecidle</span></b> goes low). Then, it will proceed to <span style="color: white;">Detect.Active</span>.</div><div><br /></div><div>In the absence of a link partner, the LTSSM will cycle between Detect.Quiet and Detect.Active with a period of approximately 12ms. This period, as well as other timeouts in PCIe, are specified with a tolerance of (+50/-0)%, so it can really be anywhere from 12-18ms. This allows for efficient logic for counter comparisons. For example, with a PHY clock of 125MHz, a count of 2^17 is 1.049ms, so a single 6-input LUT attached to <span style="color: #01ffff; font-family: courier;"><b>counter[22:17]</b></span> can just wait for 6'd12 and that will be an accurate-enough 12ms timeout trigger.</div><h4 style="text-align: left;">Detect.Active</h4><div>The transmitter for each lane attempts to detect receiver termination on that lane, indicating the presence of a link partner. This is done by measuring the time constant of the RC circuit created by the Tx AC-coupling capacitor and the Rx termination resistor. In <a href="https://docs.xilinx.com/r/en-US/pg239-pcie-phy">PG239</a>, the MAC sets the signal <b><span style="color: #01ffff; font-family: courier;">phy_txdetectrx</span></b> and monitors the result in <b><span style="color: #01ffff; font-family: courier;">phy_rxstatus</span></b> on each lane.</div><div><br /></div><div>There are three possible outcomes:</div><div><ol style="text-align: left;"><li>No receiver termination is detected on any lane. The LTSSM returns to <span style="color: white;">Detect.Quiet</span>.</li><li>Receiver termination is detected on all lanes. The LTSSM proceeds to <span style="color: white;">Polling</span> on all lanes.</li><li>Receiver termination is detected on some, but not all, lanes. In this case, the link partner may have fewer lanes. The transmitter waits 12ms, then repeats the receiver detection. If the result is the same, the LTSSM proceeds to <span style="color: white;">Polling</span> on only the detected lanes. Otherwise, it returns to <span style="color: white;">Detect.Quiet</span>.</li></ol><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Polling</h3></div><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_002.svg" width="640" /></div><div><br /></div><div>In Polling and most other states, link partners exchange Ordered Sets, easy-to-identify fixed-length packets containing link configuration information. They are transmitted in parallel on all lanes that detected receiver termination, although the contents may very per-lane in some states. The most important Ordered Sets for training are Training Sequence 1 (TS1) and Training Sequence 2 (TS2), 16-symbol packets with the following layouts:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_003.svg" style="margin-left: auto; margin-right: auto;" width="640" /></td></tr><tr><td class="tr-caption" style="text-align: center;">TS1 Ordered Set Structure<br /></td></tr></tbody></table><div style="text-align: left;"><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_004.svg" style="margin-left: auto; margin-right: auto;" width="640" /></td></tr><tr><td class="tr-caption" style="text-align: center;">TS2 Ordered Set Structure</td></tr></tbody></table><div style="text-align: left;"><br /></div><div style="text-align: left;">In the Link Number and Lane Number fields, a special symbol (PAD) is reserved for indicating that the field has not yet been configured. This symbol has a unique 8b/10b control code (K23.7) in Gen1/2, but is just defined as 8'hF7 in Gen3. Polling always happens at Gen1 speeds (2.5GT/s).</div><h4 style="text-align: left;">Polling.Active</h4><div>The transmitter sends TS1s with PAD for the Link Number and Lane Number. The receiver listens for TS1s or TS2s from the link partner.</div><div><br /></div><div>The LTSSM normally proceeds to <span style="color: white;">Polling.Configuration</span> when all of the following conditions are met:</div><div><ol style="text-align: left;"><li>Software is not commanding a transition to Polling.Compliance via the Enter Compliance bit in the Link Control 2 register.</li><li>At least 1024 TS1s have been transmitted.</li><li>Eight consecutive TS1s or TS2s have been received with Link Number and Lane Number set to PAD on all lanes, and not requesting Polling.Compliance unless also requesting Loopback (an unusual corner case).</li></ol><div>If the above three conditions are not met on all lanes after 24ms timeout, the LTSSM proceeds to <span style="color: white;">Polling.Configuration</span> anyway if at least one lane received the necessary TS1s and enough lanes to form a valid link have exited electrical idle. Otherwise, it will assume it's connected to a passive test load and go to <span style="color: white;">Polling.Compliance</span>, a substate used to test compliance with the PCIe PHY specification by transmitting known sequences.</div></div><h4 style="text-align: left;">Polling.Configuration</h4><div>The transmitter sends TS2s with PAD for the Link Number and Lane Number. The receiver listens for TS2s (<i>not </i>TS1s) from the link partner.</div><div><br /></div><div>The LTSSM normally proceeds to <span style="color: white;">Configuration</span> when all of the following conditions are met:</div><div><ol style="text-align: left;"><li>At least 16 TS2s have been transmitted <i>after receiving one TS2.</i></li><li>Eight consecutive TS2s have been received with Link Number and Lane Number set to PAD on any lane.</li></ol><div>Unlike in Polling.Active, transmitted TS are only counted after receiving at least one TS from the link partner. This mechanism acts as a synchronization gate to ensure that both link partners receive more than enough TS to clear the state, regardless of which entered the state first.</div></div><div><br /></div><div>If the above two conditions are not met after a 48ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div><div><br /></div><h3 style="text-align: left;">Configuration</h3><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_005.svg" width="640" /></div><div style="text-align: left;"><span style="font-weight: 400;"><br /></span></div><div style="text-align: left;"><span style="font-weight: 400;">The downstream-facing (host/root complex) port leads configuration, proposing link and lane numbers based on the available lanes. The upstream-facing (device/end-point) port echoes back configuration parameters, if they are accepted. The following diagram and description are from the point of view of the downstream-facing port.</span></div><h4 style="text-align: left;">Configuration.Linkwidth.Start</h4><div>The (downstream-facing) transmitter sends TS1s with a Link Number (arbitrary, 0-31) and PAD for the Lane Number. The receiver listens for matching TS1s.</div><div><br /></div><div>The LTSSM normally proceeds to <span style="color: white;">Configuration.Linkwidth.Accept</span> when the following condition is met:</div><div><ol style="text-align: left;"><li>Two consecutive TS1s are received with Link Number matching that of the transmitted TS1s, and PAD for the Lane Number, on any lane.</li></ol><div>It the above condition is not met after a 24ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div></div><h4 style="text-align: left;">Configuration.Linkwidth.Accept</h4><div>The downstream-facing port must decide if it can form a link using the lanes that are receiving a matching Link Number and PAD for the Lane Numbers. If it can, it assigns sequential Lane Numbers to those lanes. For example, an x4 link can be formed by assigning Lane Numbers 0-3.</div><div><br /></div><div>The LTSSM normally proceeds to <span style="color: white;">Configuration.Lanenum.Wait</span> when the following condition is met:</div><div><ol style="text-align: left;"><li>A link can be formed with a subset of the lanes that are responding with a matching Link Number and PAD for the Lane Numbers.</li></ol></div><div>An interesting question is how to handle a case where only some of the detected lanes have responded. Should the LTSSM wait at least long enough to handle a missed packet and/or lane-to-lane skew before exiting this state? (I don't actually know the answer, but to me it seems logical to wait for at least a few TS periods before proposing lane numbers.)</div><div><br /></div><div>If the above condition isn't met after a 2ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div><h4 style="text-align: left;">Configuration.Lanenum.Wait</h4><div>The transmitter sends TS1s with the Link Number and with each lane's proposed Lane Number. The receiver listens for TS1s with a matching Link Number and updated Lane Numbers.</div><div><br /></div><div>The LTSSM normally proceed to <span style="color: white;">Configuration.Lanenum.Accept</span> when the following condition is met:</div><div><ol style="text-align: left;"><li>Two consecutive TS1s are received with Link Number matching that of the transmitted TS1s and with a Lane Number that has changed since entering the state, on any lane.</li></ol><div>Here the spec is more explicit that upstream-facing lanes may take up to 1ms to start echoing the lane numbers, to account for receiver errors or lane-to-lane skew. So (I think) the above condition is meant to be evaluated only after 1ms has elapsed in this state.</div></div><div><br /></div><div>If the above condition isn't met after a 2ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div><h4 style="text-align: left;">Configuration.Lanenum.Accept</h4><div>Here, there are three possibilities:</div><div><ol style="text-align: left;"><li>The updated Lane Numbers being received match those transmitted on all lanes, or the reverse (if supported). The LTSSM proceeds to <span style="color: white;">Configuration.Complete</span>.</li><li>The updated Lane Numbers don't match the those transmitted, or the reverse (if supported). But, a subset of the responding lanes can be used to form a link. The downstream-facing port reassigns lane numbers for this new link and returns to <span style="color: white;">Configuration.Lanenum.Wait</span>.</li><li>No link can be formed. The LTSSM returns to <span style="color: white;">Detect</span> and starts over.</li></ol><div>Normally, lane reversal (e.g. 0-3 remapped to 3-0) would be handled by the device if it supports the feature, and its upstream-facing port will respond with matching Lane Numbers. However, if the device doesn't support lane reversal, it can respond with the reversed lane numbers to request the host do the reversal, if possible.</div></div><h4 style="text-align: left;">Configuration.Complete</h4><div>The transmitter sends TS2s with the agreed-upon Link and Lane Numbers. The receiver listens for TS2s with the same.</div><div><br /></div><div><div>The LTSSM normally proceeds to <span style="color: white;">Configuration.Idle</span> when all of the following conditions are met:</div><div><ol><li>At least 16 TS2s have been transmitted <i>after receiving one TS2</i>, on all lanes.</li><li>Eight consecutive TS2s have been received with the same Link and Lane Numbers as are being transmitted, on all lanes.</li></ol><div>If the above condition isn't met after a 2ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div></div></div><h4 style="text-align: left;">Configuration.Idle</h4><div>The transmitter sends Idle data symbols (IDL) on all configured lanes. The receiver listens for the same. Unlike Training Sets, these symbols go through <a href="https://scolton.blogspot.com/2023/08/pcie-deep-dive-part-3-scramblers-crcs.html">scrambling</a>, so this state also confirms that scrambling is working properly in both directions.</div><div><br /></div><div><div>The LTSSM normally proceeds to <span style="color: white;">L0</span> when all of the following conditions are met:</div><div><ol><li>At least 16 consecutive IDL have been transmitted <i>after receiving one IDL</i>, on all lanes.</li><li>Eight consecutive IDL have been received, on all lanes.</li></ol><div>If the above conditions aren't met after a 2ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div></div></div><div><br /></div><h3 style="text-align: left;">L0</h3><div>This is the golden normal operational state where the host and device can exchange actual data packets. The LTSSM indicates Link Up status to the upper layers of the <a href="https://scolton.blogspot.com/2023/06/pcie-deep-dive-part-2-stack-and.html">stack</a>, and they begin to do their work. One of the first things that happens after Link Up is flow control initialization by the Data Link Layer partners. Flow control is itself a state machine with some interesting rules, but that'll be for another post.</div><div><br /></div><div>But wait...the link is still operating at 2.5GT/s at this point. If both link partners support higher data rates (as indicated in their Training Sets), they can try to switch to their highest mutually-supported data rate. This is done by transitioning to <span style="color: white;">Recovery</span>, running through the Recovery speed change substates, then returning to L0 at the new rate.</div><div><br /></div><h3 style="text-align: left;">Recovery</h3><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_006.svg" width="640" /></div><div style="text-align: center;"><br /></div><div style="text-align: left;">Recovery is in many ways the most complex LTSSM state, with many internal state variables that alter state transitions rules and lead to circuitous paths through the substates, even for a nominal speed change. As with the other states, there are way too many edge cases to cover here, so I'll only focus on getting back to L0 at 8GT/s along the normal path. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">Also, since Configuration has been completed, it's assumed that Link and Lane Numbers will match in transmitted and received Training Sequences. If this condition is violated, the LTSSM may fail back to Configuration or Detect depending on the nature of the failure. For simplicity, I'm omitting these paths from the descriptions of each substate.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">When changing speeds to 8GT/s, the link must establish equalization settings during this state. In the simplest case, the downstream-facing port chooses a transmitter equalization preset for itself and requests a preset for the upstream-facing transmitter to use. The transmitter presets specify two parameters, de-emphasis and preshoot, that modify the shape of the transmitted waveform to counteract the low-pass nature of the physical channel. This can open the receiver eye even with lower overall voltage swing:</div><div style="text-align: left;"><br /></div><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_007.svg" width="640" /></div><h4 style="text-align: left;">Recovery.RcvrLock</h4><div>This substate is encountered (at least) three times.</div><div><br /></div><div>The first time this substate is entered is from L0 at 2.5GT/s. The transmitter sends TS1s (at 2.5GT/s) with the Speed Change bit set. It can also set the EQ bit and send a Transmitter Preset and Receiver Preset Hint in this state. These presets are requested values for the upstream transmitter to use after it switches to 8GT/s. The receiver listens for TS1s or TS2s that also have the Speed Change bit set.</div><div><br /></div><div>The first exit is normally to <span style="color: white;">Recovery.RcvrCfg</span> when the following condition is met:</div><div><ol style="text-align: left;"><li>Eight consecutive TS1s or TS2s are received with the Speed Change bit matching the transmitted value (1, in this case), on all lanes.</li></ol><div>The second time this subtstate is entered is from Recovery.Speed, after the speed has changed from 2.5GT/s to 8GT/s. Now, the link needs to be re-established at the higher data rate. Transitioning to 8GT/s always requires a trip through the equalization substate, so after setting its transmitter equalization, the LTSSM proceeds to <span style="color: white;">Recovery.Equalization</span> immediately.</div></div><div><br /></div><div>The third time this subtstate is entered is from Recovery.Equalization, after equalization has been completed. The transmitter sends TS1s (at 8GT/s) with the Speed Change bit cleared, the EC bits set to 2'b00, and the equalization fields reflecting the <i>downstream</i> transmitter's current equalization settings: Transmitter Preset and Cursor Coefficients. The receiver listens for TS1s or TS2s that also have the Speed Change and EC bits cleared.</div><div><br /></div><div>The third exit is normally to <span style="color: white;">Recovery.RcvrCfg</span> when the following condition is met:</div><div><ol><li>Eight consecutive TS1s or TS2s are received with the Speed Change bit matching the transmitted value (0, in this case), on all lanes.</li></ol></div><h4 style="text-align: left;">Recovery.RcvrCfg</h4><div>This substate is encoutered (at least) twice.</div><div><br /></div><div>The first time this substate is entered is from Recovery.RcvrLock at 2.5GT/s. The transmitter sends TS2s (at 2.5GT/s) with the Speed Change bit set. It can also set the EQ bit and send a transmitter preset and receiver preset hint in this state. These presets are requested values for the upstream transmitter to use after it switches to 8GT/s. The receiver listens for TS2s that also have the Speed Change bit set.</div><div><br /></div><div><div>The first exit is normally to <span style="color: white;">Recovery.Speed</span> when the following condition is met:</div><div><ol><li>Eight consecutive TS2s are received with the Speed Change bit set, on all lanes.</li></ol><div>The second time this substate is entered is from Recovery.RcvrLock at 8GT/s. The transmitter sends TS2s (at 8GT/s) with the Speed Change bit cleared. The receiver listens for TS2s that also have the Speed Change bit cleared.</div></div></div><div><br /></div><div><div>The second exit is normally to <span style="color: white;">Recovery.Idle</span> when the following condition is met:</div><div><ol><li>Eight consecutive TS2s are received with the Speed Change bit cleared, on all lanes.</li></ol></div></div><h4 style="text-align: left;">Recovery.Speed</h4><div>In this substate, the transmitter enters electrical idle and the receiver waits for all lanes to be in electrical idle. At this point, the transmitter changes to the new higher speed and configures its equalization parameters. In <a href="https://docs.xilinx.com/r/en-US/pg239-pcie-phy">PG239</a>, this is done using the <b><span style="color: #01ffff; font-family: courier;">phy_rate</span></b> and <span style="color: #01ffff; font-family: courier;"><b>phy_txeq_X</b></span> signals.</div><div><br /></div><div>The LTSSM normally returns to <span style="color: white;">Recovery.RcvrLock</span> after waiting at least 800ns and not more than 1ms after all receiver lanes have entered electrical idle.</div><div><br /></div><div>This state may be re-entered if the link cannot be reestablished at the new speed. In that case, the data rate can be changed back to the last known-good speed.</div><h4 style="text-align: left;">Recovery.Equalization</h4><div>The Recovery.Equalization substate has phases, indicated by the Equalization Control (EC) bits of the TS1, that are themselves like sub-substates. From the point of view of the downstream-facing port, Phase 1 is always encountered, but Phase 2 and 3 may not be needed if the initially-chosen presets are acceptable.</div><div><br /></div><div>In Phase 1, the transmitter sends TS1s with EC = 2'b01 and the equalization fields indicating the <i>downstream</i> transmitter's equalization settings and capabilities: Transmitter Preset, Full Scale (FS), Low Frequency (LF), and Post-Cursor Coefficient. The FS and LF values indicate the range of voltage adjustments possible for transmitter equalization.</div><div><br /></div><div>The LTSSM normally returns to <span style="color: white;">Recovery.RcvrLock</span> when the following condition is met:</div><div><ol style="text-align: left;"><li>Two consecutive TS1s are received with EC = 2'b01.</li></ol><div>This essentially means that the presets chosen in the EQ TS1s and EQ TS2s sent at 2.5GT/s have been applied and are acceptable. If the above condition is not met after a 24ms timeout, the LTSSM returns to <span style="color: white;">Recovery.Speed</span> and changes back to the lower speed. From there, it could try again with different equalization presets, or accept that the link will run at a lower speed.</div><div><br /></div><div>It's also possible for the downstream port to request further equalization tuning: In Phase 2 and Phase 3 of this substate, link partners can iteratively request different equalization settings and evaluate (via some implementation-specific method) the link quality. In a completely "known" link, these steps can be skipped if one of the transmitter presets has already been validated.</div></div><h4 style="text-align: left;">Recovery.Idle</h4><div>This substate serves the same purpose as Configuration.Idle, but at the higher data rate (assuming the speed change was successful).</div><div><div><br /></div><div>The transmitter sends Idle data symbols (IDL, 8'h00) on all configured lanes. The receiver listens for the same. These symbols now go through 8GT/s <a href="https://scolton.blogspot.com/2023/08/pcie-deep-dive-part-3-scramblers-crcs.html">scrambling</a>, so this state also confirms that 8GT/s scrambling is working properly in both directions.</div><div><br /></div><div><div>The LTSSM normally returns to <span style="color: white;">L0</span> when all of the following conditions are met:</div><div><ol><li>At least 16 consecutive IDL have been transmitted <i>after receiving one IDL</i>, on all lanes.</li><li>Eight consecutive IDL have been received, on all lanes.</li></ol><div>If the above conditions aren't met after a 2ms timeout, the LTSSM returns to <span style="color: white;">Detect</span> and starts over.</div></div></div></div><div><br /></div><h3 style="text-align: left;">LTSSM Protocol Analyzer Captures</h3><div>There are lots of places for the LTSSM to go wrong, and since it's running near the very bottom of the stack, it's hard to troubleshoot without dedicated tools like a PCIe Protocol Analyzer. In my <a href="https://scolton.blogspot.com/2023/05/pcie-deep-dive-part-1-tool-hunt.html">tool hunt</a>, I managed to get a used U4301B, so let's put it to use and look at some LTSSM captures.</div><div><br /></div><div>Side note: Somebody just scored an insane deal on a dual U4301A <a href="https://www.ebay.com/itm/375182524628">listing</a> that included the unicorn U4322A probe. If you're that someone and you want to sell me just the probe, let me know! I will take it in any condition just for the spare pins. Also, there is a reasonably-priced <a href="https://www.ebay.com/itm/186111472517?hash=item2b551bb385">U4301B</a> up right now if anyone's looking for one.</div><div><br /></div><div>But anyway, my Frankenstein U4301B + M.2 interposer is still operational and can be used with the Keysight software to capture Training Sets and summarize LTSSM progress:</div><div><br /></div><div><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_008.png" width="640" /></div><div><br /></div><div>You can see the progression through Polling and Configuration, L0, Recovery, and back to L0. In Recovery, you can see the speed change and equalization loops, crossing through the base state of Recovery.RcvrLock three times as described above.</div><div><br /></div><div>Looking at the Training Sequence traffic itself, the entire LTSSM takes around 9ms to complete in this example, with the vast majority of the time spent in the Recovery state after the speed change. Zooming in shows the details of the earlier states, down to the level of individual Training Sequences.</div><div><br /></div><div><img src="https://scolton-www.s3.amazonaws.com/img/pcie_03_009.png" width="640" /></div><div><br /></div><div>If any of the states transitions don't go as expected it's possible to look inside the individual Training Sequences to troubleshoot what conditions aren't being met. The exact timing and behavior varies a lot from device to device, though.<br /><br /></div><h3 style="text-align: left;">So you made it to L0...what next?</h3><div>L0 / Link Up means the physical link is established, so the upper layers of the PCIe stack can begin to communicate across the link. However, before any application data (memory transactions) can be transferred, the Data Link Layer must initialize flow control. PCIe flow control is itself an interesting topic that deserves a separate post, so I'll end here for now!</div><p></p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-60494045233926212522023-08-22T00:22:00.003-04:002023-08-25T09:57:01.856-04:00PCIe Deep Dive, Part 3: Scramblers, CRCs, and the Parallel LFSR<p>This post continues an exploration into the inner workings of PCIe. The <a href="https://scolton.blogspot.com/2023/06/pcie-deep-dive-part-2-stack-and.html" target="_blank">previous post</a> presented a top-level view of the PCIe Controller as a memory bus extension, with discussion of the various overheads associated with wrapping memory transfers into serial data packets. In this post, I want go to the other extreme and look at one of the low-level logic mechanisms that PCIe depends on for reliable data transfer: the parallel <a href="https://en.wikipedia.org/wiki/Linear-feedback_shift_register" target="_blank">Linear-Feedback Shift Register</a> (LFSR). This mechanism efficiently introduces randomness required to ensure DC-balanced serial data, and to validate Transaction Layer Packets (TLPs) with a <a href="https://en.wikipedia.org/wiki/Cyclic_redundancy_check">Cyclic Redundancy Check</a> (CRC).</p><h4 style="text-align: left;">PCIe 3.0 Scrambler</h4><p>PCIe signals are driven across AC-coupled differential pairs to increase immunity to noise. The transmitter and receiver may be on different boards, far apart from each other, with significant high-frequency ground offset between them. Adding series capacitors to the differential signal provides low-voltage level shifting capability to deal with this. But, this only works if the data coming across the link is DC-balanced over a data interval much shorter than the time constant formed by the AC coupling capacitor and termination resistor, which is typically 10⁴ to 10⁵ UI.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_000.svg" width="640" /></p><p>PCIe 1.0 and 2.0 use <a href="https://en.wikipedia.org/wiki/8b/10b_encoding" target="_blank">8b/10b encoding</a> to enforce DC balance. This encoding tracks the running disparity of the serial data stream and modifies 10b symbols (representing 8b data) to keep it in balance. This is also the encoding used in USB all the way up to USB 3.x Gen 1 (5Gbps), which is the same speed as PCIe 2.0. It's simple and deterministic, but it has a poor serial encoding efficiency of only 80% (8/10).</p><p>By contrast, PCIe 3.0 through PCIe 5.0 use 128b/130b encoding, where two sync bits are prepended to 128b data payloads to form 130b blocks. As discussed in the <a href="https://scolton.blogspot.com/2023/06/pcie-deep-dive-part-2-stack-and.html" target="_blank">previous post</a>, this has a much better serial encoding efficiency of 98.5% (128/130). However, the two sync bits are not sufficient to control running disparity with a 128b data payload. Instead, the data is sent through a scrambler, a Pseudo-Random Number Generator (PRNG) that remaps bits in a way that both the transmitter and receiver understand. The output stream is <i>statistically</i> DC-balanced for all real data.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_001.svg" width="640" /></p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_002.svg" width="640" /></p><p style="text-align: left;">The PCIe implementation of the PRNG for scrambling is as a Linear-Feedback Shift Register (LFSR). In the case of PCIe 3.0, the canonical implementation is a 23-bit shift register with strategically-placed XORs between some bits to instigate pseudo-randomness. The output of the shift register is then XORed with each data bit to generate the scrambled output. Each lane gets its own LFSR scrambler seeded with a different value.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_003.svg" width="640" /></p><p style="text-align: left;">This is simple logic, but it would need to run at 8GHz to be implemented in single-bit fashion like this. That's not really practical even in dedicated silicon, and is completely impossible using FPGA sequential logic. However, it's possible to parallelize the LFSR to any data width pretty easily. The key is in the name: the operation is linear, so the contributions of each bit of input data and the initial LFSR can be superimposed to generate each bit of output data and the final LFSR. This method of parallelizing the LFSR is covered very well at <a href="http://OutputLogic.com" target="_blank">OutputLogic.com</a>, with utilities to generate Verilog implementations of any LFSR and data width. I will only briefly describe the procedure here.</p><p style="text-align: left;">Using for example the 23-bit LFSR and a 32-bit data path (common for each PCIe 3.0 lane with a 250MHz PHY clock), there are a total of 23 + 32 = 55 bits that can contribute to the final LFSR and output data. Set each of those bits to one, and all other bits to zero, then run the LFSR forward by 32 steps, and record the contribution of each input bit to the output data and final LFSR. This creates a big table of bit contributions:<br /></p><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_004.svg" width="640" /></div><p></p><p style="text-align: left;">The full parallel operation is just the sum (in mod 2, so XOR) of contributions from each bit of input data and the initial LFSR. Each bit of the output data and final LFSR is the XOR combination of a specific set of input bits, with at most 55 contributing bits. On a Xilinx Ultrascale+ FPGA, wide XORs like this are easy to build using nested six-input LUTs. With two levels, you get 6² = 36 inputs. With three levels, 6³ = 216 inputs. Each level has a propagation time on the order of 1ns, so even nested three deep it's capable of running at 250MHz.<br /></p><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_005.svg" width="640" /></div><p></p><h4 style="text-align: left;">Link CRC</h4><p style="text-align: left;">Another use for the parallel LFSR is in the generation and checking of the Link CRC, a 32-bit value used for error detection in TLPs. The LCRC acts like a signature for the bits in the TLP and the value received must match the value calculated by the receiver, or the TLP is rejected. The LCRC mechanism uses a 32-bit LFSR (with XOR positions described by the standard CRC-32 polynomial 0x04C11DB7), seeded to 0xFFFFFFFF at the start of each TLP. At the end of the TLP, the 32-bit LFSR value is mapped to the packet's LCRC through some additional bit manipulation.</p><p style="text-align: left;">The LCRC operation can be parallelized in the same way as the Scrambler. The main differences are that the data is unmodified by the LCRC operation and that the data <i>does</i> contribute to the XOR sum of the next LFSR value. (This is what drives the LCRC to a unique value for each TLP.) In table form, this just changes which quadrants have XOR contributions:</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_006.svg" width="640" /></p><p style="text-align: left;">Although there are fewer rows to handle, there are now 128 + 32 = 160 columns. The LCRC is calculated on packets before they are striped across all lanes. So for a PCIe 3.0 x4 link, instead of four 32-bit data paths as in the Scrambler, there is just one 128b data path operating at 250MHz. Any of these bits and any of the 32 bits of the previous clock cycle's LFSR might contribute to the XOR for each bit of the new LFSR. This isn't a problem, though, since three levels of LUT6 can handle up to 216 XOR inputs at 250MHz, as described above.</p><p style="text-align: left;">Where things do get a little complicated is in data alignment. TLP lengths are multiples of one Double Word (DW), or 32b. So, even without considering framing, 3/4 of the possible lengths would not fit evenly into 128b data beats. Each TLP is also prepended with a 32-bit framing token (STP), the latter half of which is fed into the LCRC computation as well. So in fact all cases will involve a partial data beat.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_007.svg" width="640" /></p><p style="text-align: left;">To handle this with a 128b parallel LFSR, the LCRC mechanism must get clever. Based on the length of the packet (which is known once the STP is parsed), the 128b data window can be shifted such that the <i>last</i> data beat will be aligned with the end of the packet. This ensures that the final LFSR value can be used directly to generate the LCRC. Then, the <i>first</i> 128b data beat is padded with zeros up to the middle of the STP token, where the LCRC computation begins. (In the case of a 3DW header with no data, the first and last data beat are the same.) This creates four possible alignment cases that repeat based on the length of the TLP:</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_02_008.svg" width="640" /></p><p style="text-align: left;">Depending on the alignment case, the LFSR is seeded with a different value that accounts for the extra {16, 48, 80, 112} zeros padded onto the first data beat. These seed values are derived by seeding the reference single-bit implementation of the LFSR with 0xFFFFFFFF, then running it <i>backwards</i> for {16, 48, 80, 112} steps with zero data bits. With these seeds, the 128b parallel LFSR can be run on the zero-padded data and give the same final result as the single-bit implementation on the original data.</p><p style="text-align: left;">An interesting follow-up issue is how to handle back-to-back TLPs. Padding the first LCRC beat with zeros potentially means more than a 1:1 bit rate for the LCRC engine compared to the packet data, if there is no idle time between packets. An easy workaround could be to run two LCRC engines that take turns processing packets, although this means twice the logic area. The details are likely to vary in every implementation, so it's not something I will get into here.</p><h4 style="text-align: left;">Conclusion</h4><p style="text-align: left;">The last couple of posts were setup and background for PCIe in general. This one was more of a microscopic view of a particular logic mechanism key to several aspects of PCIe, and how it can be implemented efficiently on modern FPGAs. There are many such interesting logic puzzles to solve in gateware implementations of PCIe, and I wanted to give just one example at the lowest level I understand. I may cover other logic-level tricks in future posts, but first I think it will be more interesting to introduce what might be the scariest part of PCIe: the Link Training and Status State Machine (LTSSM). To be continued...</p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-88551258542974572602023-06-11T17:07:00.007-04:002023-06-11T19:58:29.722-04:00PCIe Deep Dive, Part 2: Stack and Efficiency<p>Before getting too caught up in the inner workings of PCIe, it's probably worth taking a look at the high-level architecture - how it's used in a system and what the PCIe controller stack looks like. PCIe is fundamentally a bi-directional memory bus extension: it allows the host to access memory on a device and a device to access memory on the host.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_000.svg" width="640" /></p><p style="text-align: left;">When a PCIe link is established between the host and a device, the host assigns address space(s) that it can use to access device memory. Optionally, it can also grant permission for the device to access portions of host system memory. In that way, the host and device memory buses are effectively connected. Each PCIe link is a point-to-point connection, but they can be combined with switches into a fabric with many devices (endpoints).</p><p style="text-align: left;">Different types of devices utilize the memory bus bridging capability of PCIe in different ways. For example, an NVMe storage device exposes only a small amount of device memory (the NVMe Controller Registers) that the host uses to configure the device and inform it when new commands have been submitted. All actual data transfer is done by the storage device reading from or writing to host memory. In this way, the NVMe storage device acts as a DMA controller between host memory and non-volatile storage.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_001.svg" style="margin-left: auto; margin-right: auto;" width="640" /></td></tr><tr><td class="tr-caption" style="text-align: center;">NVMe storage device usage of PCIe link (completion steps omitted).</td></tr></tbody></table><p style="text-align: left;">One might ask why the memory buses can't just be directly connected. For one, a native memory interface such as AXI is very wide: it might have 64-256b of data, 32-64b of address, and a bunch of control signals. This works fine inside a chip, but going from chip-to-chip, board-to-board, or across a cable, it's too many signals. The PCIe Controller encapsulates the data, address, and control signals from the memory bus into packets that can be sent across a fast serial link over a small number of differential pairs. This standard interface also allows bridging memory buses with different native interfaces, speeds, and latencies.</p><p style="text-align: left;">With that context in mind, we can look at the PCIe Controller stack, and what role each layer plays in bridging memory transactions between the host and device as efficiently and reliably as possible. The PCIe specification defines three layers: the Transaction Layer (TXL), the Data Link Layer (DLL), and the Physical Layer (PHY). These layers each have a transmit and a receive side. From the point of view of the host, the stack looks like this:</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_002.svg" width="640" /></p><p style="text-align: left;">Memory transactions from the host to the device are packaged by the TXL into a Transaction Layer Packet (TLP) with a header containing the address and other control information. The DLL prepends a framing token (STP) and appends a CRC to the TLP to create a Link Packet. This is then split into lanes and serialized by the PHY. The process happens in reverse for memory transactions from device to host, to go from serialized Link Packets back to host memory transactions.</p><p style="text-align: left;">In practice, many architectures (including Ultrascale+) break the PHY into two parts: an upper Media Access Control (MAC) layer and a lower layer still called the PHY. These are connected by the standard PHY Interface for PCI Express (PIPE), <a href="https://www.intel.com/content/www/us/en/io/pci-express/pci-express-architecture-devnet-resources.html">published by Intel</a>. It's also useful to add an explicit AXI-PCIe bridge layer above the TXL when the native memory bus is AXI, as it is in the Ultrascale+ architecture. This would be an example of what some references call the Application Layer. Expanded this way, the stack looks like this:</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_003.svg" width="640" /></p><p style="text-align: left;">Different Xilinx IPs cover different layers of the stack, as shown above. <a href="https://docs.xilinx.com/r/en-US/pg239-pcie-phy">PG239</a> (PCI Express PHY) is a low-level (PIPE down) PCIe PHY wrapper for the GTH/GTY serial transceivers. <a href="https://docs.xilinx.com/r/en-US/pg213-pcie4-ultrascale-plus">PG213</a> (UltraScale+ Devices Integrated Block for PCI Express) covers the PCIE4 hardware block that includes the TXL, DLL, and MAC layers, and interfaces to the PHY via PIPE. And <a href="AXI Bridge for PCI Express Gen3 Subsystem">PG194</a> (AXI Bridge for PCI Express Gen3 Subsystem) includes the AXI-PCIe bridge layer on top of the PCIE4 hardware block and PHY. (For Ultrascale+, this is technically implemented as a configuration of <a href="https://docs.xilinx.com/r/en-US/pg195-pcie-dma">PG195</a>, but the relevant documentation is still in PG194.)</p><p style="text-align: left;">All of these Xilinx IPs are included in Vivado at no additional cost, but not every device has the PCIE4 block(s) needed to instantiate PG213 or PG194/PG195. For the Zynq Ultrascale+ line, the <a href="https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html#productTable">product tables</a> show how many PCIe lanes are supported by integrated PCIE4 blocks for each device. In general, the larger and more expensive chips have more available PCIe hardware. But there are exceptions like the ZU6xx, ZU9xx, and ZU15xx, which have none. These can still instantiate PG239, but require a PCIe soft IP to implement the rest of the stack.</p><p style="text-align: left;">Each layer communicates with the next through a data bus that's sized to match the speed of the link. The example above is for a Gen3 x4 link, which supports 32Gb/s of serial data in each direction. In the Ultrascale+ implementation, the 250MHz clock for the 128b internal datapath is derived from the PCIe reference clock, so all layer logic is synchronous with the PHY. This seems like a perfectly-balanced data pipeline, with 32Gb/s of data coming in and going out in each direction. But in practice, overheads limit the maximum link efficiency.</p><p style="text-align: left;">First, PCIe Gen3 uses 128b/130b encoding: for each 128b serial data payload on each lane, a 2b sync header is prepended to create a 130b block. The sync bits tell the receiver whether the block is data or an Ordered Set (control sequence). In order to make room for the sync bits, PIPE requires one invalid data clock cycle in every 65-clock period.</p><p style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_004.svg" width="640" /></p><p style="text-align: left;">The period for skipping data on the 250MHz side of the PHY is 260ns, while the period for a 130b serial output block is only 16.25ns, so the PHY must implement buffering and a SERDES gearbox to make this work. The effect of the sync bits can be seen in the protocol analyzer raw data, where there are occasionally 1ns gaps in the timestamp. (The full serial data rate including sync bits would be exactly 4B/ns.) These leap-nanoseconds add up to an overall efficiency of 98.5% (64/65), as can be seen by plotting the starting timestamp of each block.</p><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_01_005.png"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_005s.png" width="640" /></a></p><p style="text-align: left;">Next, transmitters are required to periodically stop transmitting data and send a SKP Ordered Set (SKP OS), which is used to compensate for clock drift. This should happen every 370-375 blocks, and the SKP OS takes one block to transmit. Stopping the data stream also requires sending an EDS token, which may require one additional block depending on packet alignment. But even in a worst-case scenario this still represents about 99.5% (368/370) efficiency.</p><p style="text-align: left;">We can see the EDS tokens and SKP OS at regular intervals in both directions on the protocol analyzer. Interestingly, the average interval in the Host-to-Device direction is on the short side (365 blocks). Maybe it's not accounting for the 64/65 PIPE TxDataValid efficiency described above. The interval is controlled by the MAC layer, which is in PG213 in this case, so I don't think it's something that can be adjusted. The Device-to-Host direction is spot-on in this case, with a 371-block interval.</p><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_01_006.png"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_006s.png" width="640" /></a></p><p style="text-align: left;">DLLs also exchange Data Link Layer Packets (DLLPs) for Ack/Nak and flow control of TLPs. These packets are short (6B), but they must be transmitted with enough regularity to meet latency requirements and ensure receiver buffers don't overflow. There's no simple rule for when these are transmitted, only a set of constraints based on the link operating conditions. To get a feel for the typical link efficiency impact of DLLP traffic, we can look at a 100μs section of bulk data transfer and add up the combined contribution of all DLLPs:</p><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_01_007.png"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_007s.png" width="640" /></a></p><p style="text-align: left;">In total, there were 237 DLLPs transmitted in the Host-to-Device direction. Since the packets must be lane-0-aligned on an x4 link, they actually occupy 8B each. This is 1896B of overhead for nearly 400000B of data, again around 99.5% efficiency. This example is mostly unidirectional data transfer from host to device, though. If the device was also sending data to the host, there would be far more Acks going in the Host-to-Device direction. If the Ack count were similar to that of the Device-to-Host direction in this example, the efficiency would drop to around 95%.</p><p style="text-align: left;">Lastly, the biggest overhead is usually for TLP packetization. The TLP header is either 12B or 16B. The DLL adds a 4B framing token (STP) and a 4B Link CRC (LCRC). The payload size can be as high as 4096B, although it's limited to 1024B in the Ultrascale+ implementation (PG213). It's also common for devices to limit the max payload size to 128B, 256B, or 512B, depending on the capability of their PCIe Controller. This gives a range of 84.2% (128/150) to 98.1% (1024/1044) for packetization efficiency with optimally-sized transfers on Ultrascale+ hardware.</p><p style="text-align: left;">In the example capture, data is transferred from host to device in 128B-payload TLPs:</p><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_01_008.png"><img a="" src="https://scolton-www.s3.amazonaws.com/img/pcie_01_008.png" width="640" /></a></p><p style="text-align: left;">The packet has 20B of overhead for 128B of data, which would be an 86.5% efficiency. However, the host controller also inserts 12B of logical idle (zeros) to align the next STP token to the start of a block. This isn't required by the PCIe protocol, but may be inherent in the implementation of the controller. For this payload size, it drops the efficiency to 80% (128/160). </p><p style="text-align: left;">That packetization efficiency dominates the overall link efficiency, which hovers between 75% and 80% during periods of stable data transfer:<br /></p><div style="text-align: center;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_01_009.svg" width="640" /></div><p></p><p style="text-align: left;">In this case, increasing the max payload size would have the most positive impact on throughput. PG213 can go up to 1024B, but the device controller may be the limiting factor.</p><p style="text-align: left;">In PCIe 6.0, a big change will be introduced that removes sync bits and consolidates DLLPs, framing tokens, and the LCRC into a fixed 20B overhead in each 256B unit (called a FLIT, for Flow Control unIT). This implies a fixed 92.2% efficiency for everything other than the SKP OS and TLP header overhead, and also a fixed latency for Ack/Nak and flow control, a nice simplification.</p><p style="text-align: left;">But for now we're still in the realm of PCIe Gen3, where we can expect an overall link efficiency in the 75-95% range, depending on the variety of factors described above as well as details of the controller implementations.</p><p style="text-align: left;">The packetization and flow control functions described above are the domain of the Transaction Layer and Data Link Layer, but there are also some really interesting functions of the MAC and PHY layers that facilitate reliable serial data transfer across the physical link. These will have to be topics for one or more future posts, though.</p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-36707802985377862362023-05-07T20:20:00.010-04:002023-11-26T11:24:28.257-05:00PCIe Deep Dive, Part 1: Tool Hunt<p>Over the past few years, I've been <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">developing</a> and <a href="http://scolton.blogspot.com/2021/10/zynq-ultrascale-bare-metal-nvme-2gbs.html">improving</a> very fast standalone NVMe-based storage capability for the Zynq Ultrascale+ architecture, to keep up with the absurd speeds of modern SSDs. (Drives like the <a href="https://www.tomshardware.com/reviews/seagate-firecuda-530-m2-nvme-ssd-review/2">Seagate Firecuda 530</a> and <a href="https://www.tomshardware.com/reviews/sabrent-rocket-4-plus-g-ssd-review/2">Sabrent Rocket 4 Plus-G</a> can now hit 3GB/s+ <i>sustained</i> <a href="https://www.reddit.com/r/NewMaxx/wiki/basics/#wiki_number_of_levels_or_bits_per_cell">TLC</a> write speeds, with much higher <a href="https://www.reddit.com/r/NewMaxx/wiki/basics/#wiki_slc_cache_type">pSLC cache</a> peaks.) But my knowledge pretty much ended at the interface to the Xilinx DMA/Bridge Subsystem for PCI Express (<a href="https://docs.xilinx.com/r/en-US/pg194-axi-bridge-pcie-gen3/Introduction">PG194</a>/<a href="https://docs.xilinx.com/r/en-US/pg195-pcie-dma">PG195</a>). In the usual fashion, I'm now going to dive deeper to explore in more detail how the AXI-PCIe bridge works, and what the PCIe stack actually looks like.</p><p>Something I found interesting about PCIe in general is that there seems to be a pretty large barrier built up around the black box. Even just finding learning resources is much harder than it should be. The best I found was <a href="https://www.mindshare.com/Books/Titles/PCI_Express_Technology_3.0">PCI Express Technology 3.0</a> and some accompanying material by MindShare, but even that seems like a prose wrapper on top of the specification. There isn't anything that I would consider a beginner's guide, like you might find for USB or Ethernet.<br /></p><p><span style="color: #990000;">[Edit by Future Shane] There is a very good series of four articles from Simon Southwell starting <a href="https://www.linkedin.com/pulse/pci-express-primer-1-overview-physical-layer-simon-southwell/?trackingId=Ei0VpssCS3anOPhl0i1l6Q%3D%3D">here</a> that offers a thorough introduction to PCIe. Definitely check it out if you're going to be exploring PCIe.</span></p><p>For physical tools, the situation is even more bleak. The speeds in PCIe Gen3 (8GT/s) put it in the range where an oscilloscope that can actually measure the signal will cost more than a car. But for all but the lowest-level hardware debugging, a digital capture would suffice, and that's where a protocol analyzer would be nice. Unfortunately, there is no Wireshark equivalent for PCIe; protocol analyzers for it are dedicated hardware that only a few companies develop, and they are priced astronomically.</p><p>That is...unless you scout them on eBay for a year.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_00_000.jpg" style="margin-left: auto; margin-right: auto;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_00_000s.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Biggest "that escalated quickly" of my test equipment stack (ref. PicoScopes below table).</td></tr></tbody></table><p>This is a used <a href="https://www.keysight.com/us/en/product/U4301B/pci-express-protocol-analyzer.html">U4301B</a> that I got in what has to be my second-best eBay score of all time, for less than it would have cost me to rent one for a month. There are only ever a handful of them up for auction at any given time, and the market is so small that the price is basically random, so if you're actually looking for one I can only wish you luck. This one goes up to Gen3 x8, which is fine for my purposes. If you only need Gen1/2 capability, the situation is much better.</p><p><span style="color: #990000;">[Edit by Future Shane] There is one <a href="https://www.ebay.com/itm/186111472517">listed on eBay</a> for a good price right now if anyone else is looking for one. (I'll remove this note after it's no longer available.)</span></p><p>The U4301B is actually just the instrument in the bottom slot of the <a href="https://www.keysight.com/us/en/product/M9505A/axie-5-slot-chassis.html">M9505A</a> AXIe Chassis. This is meant to connect to a PCIe slot on a host machine using an <a href="https://www.molex.com/en-us/products/connectors/high-speed-pluggable-io/ipass-connector-system">iPass</a> cable and <a href="https://onestopsystems.com/collections/pcie-cable-adapters/products/pcie-x4-gen2-host-cable-adapter">interface card</a>. Newer versions of the chassis controller have a laptop-friendly Thunderbolt connection instead. I "upgraded" mine using an eGPU enclosure, the smaller black box sitting on top.</p><p>I said that the U4301B was my second-best eBay score of all time, and that's because the number one is the <a href="https://www.keysight.com/us/en/product/U4322A/pcie-mid-bus-probe.html">U4322A</a> probe that I got to go with it, from a different auction. The protocol analyzer is useless without a probe or interposer, and those are even harder to find used. I have <i>never</i> seen a U4322A on eBay before or since the one I got, and all other online listings for them are dead-ends. So the fact that I got one for what might as well be free compared to the new cost is just plain luck.<br /></p><p>It was, however, a lot broken...</p><p>The probe has two rows of spring-loaded contacts that are meant to touch down on test pads for the PCIe signals. Unfortunately, mine was missing several pins and many others were bent or broken. It had been treated like a scrap cable, rather than a delicate probe. No problem, though, I can just replace the spring pins with some equivalent Mill-Max parts...<br /></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_00_001.jpg" style="margin-left: auto; margin-right: auto;"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_00_001s.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">...oh, well shit.</td></tr></tbody></table><p>This was one of the most ridiculous things I have ever seen under the microscope. Each spring pin has a <i>surface-mount resistor </i>soldered into its tip, and encased in epoxy. What the multi-GHz fuck is going on with these? Well, I suspect they each make up part of a passive probe, also called a Low-Z or Z0 probe. This <a href="https://www.youtube.com/watch?v=eZhMIR0l3xU">video</a> explains the concept in detail; it's forming a resistive divider with the 50Ω termination. But it must have extremely low capacitance on the input side of the resistor, hence the resistors embedded in the tips. The good news is that there are no amplifiers in the probe head, so there's not much else that can be broken.</p><p>There's no replacement for these pins, so the ones that were missing or broken were a lost cause. But luckily there were enough intact ones to make a full bidirectional x4 link, which is all I really needed. They weren't all in the right locations, so I had to carefully rearrange them with a soldering iron, taking care to use as little solder as possible while still making a strong connection. After making the x4 link, there are only a couple of spare pins remaining, so I need to be very careful with this probe.</p><p>Actually the U4322A was not my first choice; what I really wanted was a <a href="https://www.keysight.com/us/en/product/U4328A/m-2-pci-express-interposer-socket-3-m-key.html">U4328A</a> M.2 interposer, which taps off the signals at an M.2 connector bridge. But I can convert my basically free U4322A into that using a basically free circuit board. This board just has the test pad footprint for the U4322A in between a short M.2 extension. I carefully mounted the U4322A to the board with standoffs and don't really intend to ever take it off again.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_00_002.png"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_00_002s.png" width="640" /></a></div><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_00_003.jpg"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_00_003s.jpg" width="640" /></a></p><p>Somewhat to my surprise, this collection of parts actually does work. I was worried that there would be some license nonsense involved, but the instrument license seems to go with the instrument. The <a href="https://www.keysight.com/us/en/lib/software-detail/instrument-firmware-software/logic-and-protocol-analyzer-software-64bit-2511828.html">host software</a> doesn't require a separate license and worked right away, even through my weird Thunderbolt eGPU enclosure hack. And that's really where the value is. It wouldn't be hard to make an in-system <a href="https://cdrdv2.intel.com/v1/dl/getContent/643108">PIPE</a> traffic logger on a Zynq Ultrascale+, and I might do that anyway, but parsing and visualizing the data in a convenient way takes a lot of effort. With the LPA Software, you just get nice graph and packet views straight away:</p><p style="text-align: center;"><a href="https://scolton-www.s3.amazonaws.com/img/pcie_00_004.png"><img src="https://scolton-www.s3.amazonaws.com/img/pcie_00_004s.png" width="640" /></a></p><p>This all seems like a lot of effort for probing an interface that's now at least two generations old. All this equipment is outdated and could for sure be replaced with a single-board interposer based on a Zynq Ultrascale+. All it needs is two GTH quads, a bunch of RAM, and a high-speed interface to the outside world. But I don't think Keysight or Teledyne LeCroy are interested in that - Gen5 is where the money is. Interestingly, though, the new <a href="https://www.keysight.com/us/en/product/P5552A/p5552a-pcie-5-0-protocol-analyzer.html">Keysight Gen5 analyzer</a> <i>is</i> a single-board interposer.</p><p>But for now I have Gen3 protocol analysis capability, which is good enough for my purposes. I've used it a bunch in the past few months to explore the different layers of the PCIe stack and components within. There are some really interesting parts that I may cover in future posts. But I'll probably start with an overview of the whole stack, and where the available Xilinx IPs fit into it, since even that is a little confusing at first. There are hard and soft (i.e. HDL) components to it, and not every device has an out-of-the-box solution for making the whole stack. That's enough material for an entire post though, so I'll end this one here.</p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-60005885806858975482021-10-09T10:16:00.001-04:002021-11-23T16:32:05.400-05:00Zynq Ultrascale+ Bare Metal NVMe: 2GB/s with FatFs + exFAT<p>This is a quick follow-up to my <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">original post</a> on speed testing bare metal NVMe with the Zynq Ultrascale+ AXI-PCIe bridge. There, I demonstrated a lightweight NVMe driver running natively on one Cortex-A53 core of the ZU+ PS that could comfortably achieve >1GB/s write speeds to a suitable M.2 NVMe SSD, such as the Samsung 970 Evo Plus. That's without any hardware acceleration: the NVMe queues are maintained in external DDR4 RAM attached to the PS, by software running on the A53.</p><p>I was actually able to get to much higher write speeds, over 2.5GB/s, writing directly to the SSD (no file system) with block sizes of 64KiB or larger. But this only lasts as long as the SLC cache: Modern consumer SSDs use either TLC or QLC NAND flash, which stores three or four bits per cell. But it's slower to write than single-bit SLC, so drives allocate some of their free space as an SLC buffer to achieve higher peak write speeds. Once the SLC cache runs out, the drive drops down to a lower sustained write speed.</p><p>It's not easy to find good benchmarks for sustained sequential writing. The best I've seen are from <a href="https://www.tomshardware.com/">Tom's Hardware</a> and <a href="https://www.anandtech.com/">AnandTech</a>, but only as curated data sets in specific reviews, not as a global data set. For example, this <a href="https://www.tomshardware.com/reviews/sabrent-rocket-4-plus-m2-nvme-ssd-review/2">Tom's Hardware review of the Sabrent Rocket 4 Plus 4TB</a> has good sustained sequential write data for competing drives. And, this <a href="https://www.anandtech.com/show/16087/the-samsung-980-pro-pcie-4-ssd-review/3">AnandTech review of the Samsung 980 Pro</a> has some more good data for fast drives under the Cache Size Effects test. My own testing with some of these drives, using ZU+ bare metal NVMe, has largely aligned with these benchmarks.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFbCSBuIDiDMRj1HJHdfma8XoPLWgT4MY-gMq_2J6-sStmWhcFFt53UIT2uQunCR4QwXTANIeWdEQf7Vru85ANMKm36J3iioGbBjNMiDtu-9KZ8jcH2OHoswficSDORFi2s3q48fkLSXs/s1585/CompareFOB.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="893" data-original-width="1585" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFbCSBuIDiDMRj1HJHdfma8XoPLWgT4MY-gMq_2J6-sStmWhcFFt53UIT2uQunCR4QwXTANIeWdEQf7Vru85ANMKm36J3iioGbBjNMiDtu-9KZ8jcH2OHoswficSDORFi2s3q48fkLSXs/w640-h360/CompareFOB.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"></td></tr></tbody></table><p>The unfortunate trend is that, while peak write speeds have increased dramatically in the last few years, sustained sequential write speeds may have actually gotten <i>worse</i>. This trend can be seen globally as well as within specific lines. (It might even be true within <a href="https://www.tomshardware.com/news/samsung-is-swapping-ssd-parts-too">different date codes of the same drive</a>.) Take for example the <a href="http://images.anandtech.com/doci/16087/seq-fill-970pro-1024.png">Samsung 970 Pro</a>, an MLC (two bit per cell) drive released in 2018 that had no SLC cache but could write its full capacity (1TB) MLC at over 2.5GB/s. Its successor, the <a href="https://www.anandtech.com/show/16087/the-samsung-980-pro-pcie-4-ssd-review/3">980 Pro</a>, has much higher peak SLC cache write speeds, nearing 5GB/s with PCIe Gen4, but dips down to below 1.5GB/s at some points after the SLC cache runs out.</p><p>Things get more complicated when considering the allocation state of the SSD. The sustained write benchmarks are usually taken after the entire SSD has been deallocated, via a secure erase or whole-drive TRIM. This restores the SLC cache and resets garbage collection to some initial state. If instead the drive is left "full" and old blocks are overwritten, the SLC cache is not recovered. However, this may also result in faster and more steady sustained sequential writing, as it prevents the undershoot that happens when the SLC cache runs out and must be unloaded into TLC.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjEA2II-xL_12nxm8NlZvsMk-tTEaZYOCE9SfpaxZok8ZkZ5LXFI4_vmXI2q-RWxa39TiLXRAjls8gdN6iJAXfvTbUgVEmQXkSmOxYVfCQS6wr6Uw66XOfllb2hlLklfatBztdu-7Jiuw/s1601/CompareTRIM.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="897" data-original-width="1601" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjEA2II-xL_12nxm8NlZvsMk-tTEaZYOCE9SfpaxZok8ZkZ5LXFI4_vmXI2q-RWxa39TiLXRAjls8gdN6iJAXfvTbUgVEmQXkSmOxYVfCQS6wr6Uw66XOfllb2hlLklfatBztdu-7Jiuw/w640-h358/CompareTRIM.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"></td></tr></tbody></table><p>So in certain conditions and with the right SSD, it's just possible to get to sustained sequential write speeds of 2GB/s with raw disk access. But, what about with a file system? I originally tested <a href="http://elm-chan.org/fsw/ff/00index_e.html">FatFs</a> with the drive formatted as FAT32, reasoning (incorrectly) that an older file system would be simpler and have less overhead. But as it turns out, exFAT is a much better choice for fast sustained sequential writing.</p><p>The most important difference is how FAT32 and exFAT check for and update cluster allocation. Clusters are the unit of memory allocated for file storage - all files take up an integer number of clusters on the disk. The clusters don't have to be sequential, though, so the File Allocation Table (FAT) contains chained lists of clusters representing a file. For sequentially-written files, this list is contiguous. But the FAT allows for clusters to be chained together in any order for non-contiguous files. Each 32b entry in the FAT is just a pointer to the next cluster in the file.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCJOhHf2yz_c9_cQ6KNbSZnEYNmnfhOYBsY52Apos-zAa6yZ5-UOwUUYwDAic2GRwd7D1xPudAoNip0w_jxgyCvpqWYp6iAGcqfbUZmr9n_zzE2cPshQqEc2ONv_k5FSIpCi6gvCP-RHc/s1307/e74.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1014" data-original-width="1307" height="497" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCJOhHf2yz_c9_cQ6KNbSZnEYNmnfhOYBsY52Apos-zAa6yZ5-UOwUUYwDAic2GRwd7D1xPudAoNip0w_jxgyCvpqWYp6iAGcqfbUZmr9n_zzE2cPshQqEc2ONv_k5FSIpCi6gvCP-RHc/w640-h497/e74.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">FAT32 cluster allocation entirely based on 32b FAT entries.</td></tr></tbody></table><p>In FAT32, the cluster entries are mandatory and a sequential write must check and update them as it progresses. This means that for every cluster written (64KiB in maxed-out FAT32), 32b of read and write overhead is added. In FatFs, this gets buffered until a full LBA (512B) of FAT update is ready, but when this happens there's a big penalty for stopping the flow of sequential writing to check and update the FAT.</p><p>In exFAT, the cluster entries in the FAT are optional. Cluster allocation is handled by a bitmap, with one bit representing each cluster (0 = free, 1 = allocated). For a sequential file, this is all that's needed. Only non-contiguous files need to use the 32b cluster entries to create a chain in the FAT. As a result, sequential writing overhead is greatly reduced, since the allocation updates happen 32x less frequently.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7RT57TasiKoEUEDvcBYRvyubADyNZeU_KAFYorYvhK-nXdyFB4ubBf4wR9VSzCSTJ1LRutcY6aVZnUJlk8Ux1GZmuT3u76ORq7ESoiswn6eTnfln7BdEGHW6T2fNfLPZEdk43F1c7uJM/s1313/e75.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1204" data-original-width="1313" height="586" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7RT57TasiKoEUEDvcBYRvyubADyNZeU_KAFYorYvhK-nXdyFB4ubBf4wR9VSzCSTJ1LRutcY6aVZnUJlk8Ux1GZmuT3u76ORq7ESoiswn6eTnfln7BdEGHW6T2fNfLPZEdk43F1c7uJM/w640-h586/e75.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">exFAT cluster allocation using bitmap only for sequential files.</td></tr></tbody></table><p>The cluster size in exFAT is also not limited to 64KiB. Using larger clusters further reduces the allocation update frequency, at the expense of more dead space between files. If the plan is to write multi-GB files anyway, having 1MiB clusters really isn't a problem. And speaking of multi-GB files, exFAT doesn't have the 4GiB file size limit that FAT32 has, so the file creation overhead can also be reduced. This does put more data "at risk" if a power failure occurs before the file is closed. (Most of the data would probably still be in flash, but it would need to be recovered manually.)</p><p>All together, these features reduce the overhead of exFAT to be almost negligible:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghxUYlV4AANFtU9pMZIMLT6QLDdegR0qjV7AxsULmLDAsAbVNuoei5S7fHgohufFDmFAENEla-1PiVdD6mkTAYTUIAQ1ZUtwjKUaYkvtB9Y-ICFxz9pHNrfKLc25ZmeoADStG9Bn2WLDE/s1601/CompareFS.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="897" data-original-width="1601" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghxUYlV4AANFtU9pMZIMLT6QLDdegR0qjV7AxsULmLDAsAbVNuoei5S7fHgohufFDmFAENEla-1PiVdD6mkTAYTUIAQ1ZUtwjKUaYkvtB9Y-ICFxz9pHNrfKLc25ZmeoADStG9Bn2WLDE/w640-h358/CompareFS.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"></td></tr></tbody></table><p>With 1MiB clusters and 16GiB files, it's possible to get ~2GB/s of sustained sequential <i>file</i> writing onto a 980 Pro for its entire 2TB capacity. I think this is probably the fastest implementation of FatFs in existence right now. The data block size still needs to be at least 64KiB, to keep the driver overhead low. But if a reasonable amount of streaming data can be buffered in RAM, this isn't too much of a constraint. And of course you do have to keep the SSD cool.</p><p>I've updated the bare metal NVMe test project to Vivado/Vitis 2021.1 <a href="https://github.com/coltonshane/SSD_Test">here</a>. It would still require some effort to port to a different board, and I still make no claims about the suitability of this driver for any real purposes. But if you need to write massive amounts of data and don't want to mess around in Linux (or want to try something similar in Linux user space...) it might be a good reference.</p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com2tag:blogger.com,1999:blog-8200098102909041178.post-46088207142664736942021-09-12T23:04:00.003-04:002021-09-20T11:33:29.833-04:00TinyCross: New UI and Front Wheel Traction Control<p> In the <a href="http://scolton.blogspot.com/2021/08/tinycross-4wd-80a-data-logging.html">last post</a>, I finally did some actual data logging with TinyCross set up in 4WD, 80A peak per motor, which is the rated current. Based on <a href="http://scolton.blogspot.com/p/cap-kart.html#tinykart">tinyKart</a>, I know they can handle a a bit more for short durations, maybe even up to 120A. But the data logs (and many instances of having rocks flung into my face) demonstrate that the front wheels reach their traction limit somewhere around 60A on asphalt.</p><p>The behavior of front wheel slip on a go-kart is something new to me. In a straight line, the initiation of the slip and the acceleration of the wheel actually isn't the biggest problem. It's when the wheel regains traction and slows down that bad things happen. The restored grip combines with the energy being dumped from the wheel's moment of inertia to generate a quick pulse of torque on that side, which creates a lot of torque steer.</p><p>To deal with this, I wanted to implement some form of traction control, at least for the front wheels, so that I could get the most torque out of them as possible without the steering disturbances and rock shooting. But first, I needed a way to easily configure both the motor currents and the traction control settings without having to drag around my laptop everywhere. So, I finally built out the steering wheel UI to include a bunch of settings:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZaDxcsaIpz4ijBS9ZGksAvbhjJVqzm7PVBxjBntWNA_4lHPtN1z2pONUNFMByo3tnfbuAANcWixlqZ2iidWCE5RZURevvhMOvJmtNjFQSBeemJQaQA-vAQ1ndv4EJwC5_aN5ZUcrtlds/s2048/tc98.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZaDxcsaIpz4ijBS9ZGksAvbhjJVqzm7PVBxjBntWNA_4lHPtN1z2pONUNFMByo3tnfbuAANcWixlqZ2iidWCE5RZURevvhMOvJmtNjFQSBeemJQaQA-vAQ1ndv4EJwC5_aN5ZUcrtlds/w640-h480/tc98.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Sorry for the exposure; it's the only way to capture the full OLED refresh period.</td></tr></tbody></table><p style="text-align: left;">Anyone familiar with the <a href="https://freeflysystems.com/movi-controller">MōVI Controller</a> might recognize the OLED display. I chose this for daylight visibility and responsiveness (~50Hz update rate). The menu interface is essentially the same as the one I built the day before NAB 2014... The left knob scrolls through the menu. The right knob adjust settings and, by clicking or holding, performs actions.</p><p style="text-align: left;">In the four corners are three motor parameters for the corresponding motors: S for Status, which shows error codes. F for Forward peak current, and R for Reverse (braking, or actually reversing) peak current. Setting both to zero masks out the CAN command from that motor, triggering a timeout that turns off the gate drivers entirely. A click and hold on S triggers an encoder recalibration for that motor.</p><p style="text-align: left;">In the second column from the left, the first three settings relate to data logging: LS for Logger Status, FN for File Number (click to start a new file), and LT for Logger Time, the time in [ms] for a single row of the data log to be written. Then, there are two parameters for tuning traction control: TT for Traction Threshold, and TG for Traction Gain, which I will explain shortly.</p><p style="text-align: left;">The reason I wanted to be able to adjust peak currents from the steering wheel is because I agree with this early Tesla <a href="https://www.tesla.com/blog/spin-stops-here">blog post</a>: "...it's much safer to avoid wheelspin altogether than react to it." If I know the surface supports front wheel current around 60A, there's not much point in setting it higher than that. But, I want to be able to set it higher for testing, or adjust it for different surfaces.</p><p style="text-align: left;">As for the traction control itself, there are a lot of corner cases to think about in 4WD, but the main problem I'm trying to solve is front wheel slip. If I assume the rear wheels are not slipping, then I can use their average speed as a reference. From there, it's <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhn1AoiTrgVGp20NRQykuhhg999lD3uTm_jY7MSHTQ7IdnIjp6U1ggplaSq0j3KBjGTNY2LnWpcvAAcZDXdxGz35qYiHxNJQbFyb40acbHuepXU3z84tindKb53HViEqX4Baa5UCoIgkd4/w399-h640/launch_4WD_80A_12V.png">easy to see</a> if a front wheels is running faster than that reference, and reduce the current to that motor if so. This only needs two settings: a Traction Threshold (TT) that sets how much wheel slip is allowed, and a Traction Gain (TG) that sets how much to reduce the current per unit slip above the threshold. The Traction Threshold prevents overactuation in normal conditions and allows for speed differential due to turning radius.</p><p style="text-align: left;">But what happens if a rear wheel does slip? Well, then the front wheel might slip too. At that point, I'm probably in some kind of a four wheel sideways drift anyway, so alternate control laws are going to apply. Being able to trigger some rear wheel slip with the throttle is part of the fun, too, so having complete 4WD traction control isn't something I necessarily need to solve.</p><p style="text-align: left;">With the new UI setup and the simple front wheel traction control in place, it was time to do some tuning...</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkTVDhMfdtneAJ-oxHkQih4OTOGI_exkprfMSNbxENdcJgBA-2ksvHU3Fv5z8JofVnOr2w7h9PTnksJul3JKESimV-quGWPMuILKMIyiR04bo9dFdOtfYAd74IJHNsom-9JDFoOMHZ8Es/s2048/tc95.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkTVDhMfdtneAJ-oxHkQih4OTOGI_exkprfMSNbxENdcJgBA-2ksvHU3Fv5z8JofVnOr2w7h9PTnksJul3JKESimV-quGWPMuILKMIyiR04bo9dFdOtfYAd74IJHNsom-9JDFoOMHZ8Es/w640-h480/tc95.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">...or not.</td></tr></tbody></table><p style="text-align: left;">At first, everything seemed to be going okay. I did a couple of runs at 60A front current and 80A rear current and the traction control seemed to be working as intended. But then during light regenerative braking at around 30mph, I heard the all-too-familiar sound of a FET popping, followed by some more bad noises and smells from the front drive. Upon inspection, only two FETs actually died, but they also took out many of the power traces, meaning this board was trash.</p><p style="text-align: left;">So what happened? Well, unfortunately, the data log was not very helpful in this case. It did show the speed (30mph) and current command (around -10A), but nothing out of the ordinary up until the point of failure. There is only one data point showing a Q-Axis current of 286A on the front left motor, followed by an undervoltage fault, which might have been the battery sagging or the power input traces getting blown up. So whatever happened, happened quick.</p><p style="text-align: left;">It's been a while since I've actually destroyed a motor controller, so I was a little disappointed. But after some thought, I didn't think this was due to the new traction control stuff. That's only applied during acceleration, and this failure definitely happened under braking. I think it's more likely that the front left motor just lost sync and the back EMF at 30mph was high enough to do damage. Up until now, I have only had a relatively slow overcurrent limit of 160A (or more) for 10ms. These FETs have a pretty insane Safe Operating Area (SOA), but that limit does leave room for exceeding it with currents above 400A:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiljq7fFCCpyVCo6V6K5Khy4JQ6RZO9YOUWOydWylawIPNHeAOVwTFAkk52FWXKu-N8UizhJ761FrCSQEi-NdqNY0DbC4qk6AspuddTl0XUUGZylWhvybM_Q7ZS1KF8ar2NYauDmCqsiEI/s1072/tc99.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="956" data-original-width="1072" height="356" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiljq7fFCCpyVCo6V6K5Khy4JQ6RZO9YOUWOydWylawIPNHeAOVwTFAkk52FWXKu-N8UizhJ761FrCSQEi-NdqNY0DbC4qk6AspuddTl0XUUGZylWhvybM_Q7ZS1KF8ar2NYauDmCqsiEI/w400-h356/tc99.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"></td></tr></tbody></table><p style="text-align: left;">This system could easily generate a 400A transient if a motor loses sync at 30mph. And the motor position and speed data <i>does</i> cut out at the same data point as the failure. But that's not enough to determine cause and effect. So for now I can only make changes that might help and hope for the best. I added in several more stages of faster overcurrent protection, up to 300A for a single ADC/PWM cycle (42.7μs). These overlap enough to cover the entire R_DS(on)-limited boundary of the SOA (up to the pulse rating of 1450A for 100μs!).</p><p style="text-align: left;">A faster overcurrent trip doesn't help with whatever caused the motor to lose sync in the first place (if that is what happened). I have seen at least a couple previous instances where the encoders, which supply emulated Hall effect sensor signals, have behaved as if they were completely reset. Even though I only use the buffered and optically isolated virtual Hall effect sensor signals for commutation, I was still reading the SPI data anyway. Maybe a SPI read got corrupted by noise and turned into a write that either reconfigured or entirely reset the encoder mid-run? To protect against this, I now disabled the SPI transactions entirely other than during initialization and calibration.</p><p style="text-align: left;">So with these changes and my last and only spare drive, I went back out for another try. This time, I ran into no motor drive issues and was actually able to test and tune the front wheel traction control as I originally intended. The difference is immediately obvious while driving and in the data. First, a test at 80A front, 90A rear, with no traction control:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAxiD0tyB102XD4J4HqUJet6lb1_Ro033hW2ijFgSNvVz5VLpSGuz8MZGbcYpjjr7eg-DmXcdcNzcmmhE211F6piZUrc3n3DEQD9_XCfpKiXfiDPqJi9ZIq2MujYFyDVKfcXninDeoXrQ/s1741/tc00.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1741" data-original-width="1369" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAxiD0tyB102XD4J4HqUJet6lb1_Ro033hW2ijFgSNvVz5VLpSGuz8MZGbcYpjjr7eg-DmXcdcNzcmmhE211F6piZUrc3n3DEQD9_XCfpKiXfiDPqJi9ZIq2MujYFyDVKfcXninDeoXrQ/w504-h640/tc00.png" width="504" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Front wheel traction control off.</td></tr></tbody></table><p style="text-align: left;">As before, the front right wheel starts slipping at about 60A and spins up to 2-3x the actual ground speed. The front right always seems to lose grip first, a mystery to solve another day. When I let off the throttle and it regains traction, the torque pulse creates substantial torque steer, jerking the steering wheel almost 20º to the left, which I then have to counteract immediately to stay on course. Overall, it's impossible to sustain peak acceleration for more than a second or so before having to deal with the wheel spin and torque steer.</p><p style="text-align: left;">And now with the same currents, but front wheel traction control on:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEideQlOdqmAa1EWKA-uT3j_O9unV8W5QK1VwrEX0_IKyHNHj82C8K6u-ORRj7cycxhA2c1jj1HlQz4cRL4Mn1VIrCh-wgh6avaQLHxVhKh5iMkYAM5Dlf8e_2kSSobn83OcMVjRYILi-pk/s1731/tc01.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1731" data-original-width="1369" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEideQlOdqmAa1EWKA-uT3j_O9unV8W5QK1VwrEX0_IKyHNHj82C8K6u-ORRj7cycxhA2c1jj1HlQz4cRL4Mn1VIrCh-wgh6avaQLHxVhKh5iMkYAM5Dlf8e_2kSSobn83OcMVjRYILi-pk/w506-h640/tc01.png" width="506" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Front wheel traction control on.</td></tr></tbody></table><p style="text-align: left;">The front right (FR) current now averages a bit below 60A and its speed is held to just a small margin above the actual ground speed. It's never able to build up momentum and then "catch", inducing torque steer. This allows continuous acceleration up to and past 30mph. The front left (FL) also starts to slip in the 20-30mph range, but the traction control catches it too. The overall result is a much more controllable launch and far fewer rocks being thrown up by the front wheels.</p><p style="text-align: left;">After finding traction control settings that I liked, I switched back to current settings that more closely match the actual traction limits: 60A front and 100A rear. This still gives a reasonable 0.45g launch, but with less likelihood of triggering the traction control on asphalt. I'd like to push to >0.5g, to match tinyKart's most extreme configuration, but that'll either require 120A on the rear or changing the gear ratio a bit. At 60A / 100A, the front motors still share enough of the load that the rear motors stay at healthy temperature after some acceleration runs:</p><p style="text-align: left;"></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJYQ-19H8n1XDjK4_o979xE8Bd63OEeUhvRonZ-SwyZeOvLSLnzJo98s7Jbuw8tZNre98CA6aBBSJMl39VB4rJK5GeBOM49QZ-m5VeFarN9LSMv1_7vDZf7cgmfHSAr-GcyGzjCgcxQc4/s1440/tc97.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1080" data-original-width="1440" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJYQ-19H8n1XDjK4_o979xE8Bd63OEeUhvRonZ-SwyZeOvLSLnzJo98s7Jbuw8tZNre98CA6aBBSJMl39VB4rJK5GeBOM49QZ-m5VeFarN9LSMv1_7vDZf7cgmfHSAr-GcyGzjCgcxQc4/w640-h480/tc97.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Rear motors are doing most of the work, but...</td></tr></tbody></table><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie37dGZ135or-3iC1gCtxzbXjd3zvQIamTUoZLWnv6RXzp0b4dKVp55StYYm_V4AZy4sgQ15OfzFTAyLC4G2vnuKo7X1GkmbNUooab6WigDK7V4J5Y5IeBbmAQvyhHAIYfGD-r3RxOutk/s1440/tc96.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1080" data-original-width="1440" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie37dGZ135or-3iC1gCtxzbXjd3zvQIamTUoZLWnv6RXzp0b4dKVp55StYYm_V4AZy4sgQ15OfzFTAyLC4G2vnuKo7X1GkmbNUooab6WigDK7V4J5Y5IeBbmAQvyhHAIYfGD-r3RxOutk/w640-h480/tc96.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">...they are at a reasonable temperature.</td></tr></tbody></table><br />And finally I did some less structured testing by just driving through the gravel corner in my parking lot and intentionally adding throttle to induce slip. It behaves pretty well, slipping and oversteering about the right amount to be controllable but still fun:<div><br /></div><div><iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/EdQ26vxMblk" title="YouTube video player" width="640"></iframe></div><div><br /></div><div>I think at this point most of the handling bottlenecks are back on the mechanical side. There's a small amount of backlash in the steering column that definitely exaggerates the residual torque steer, especially at high speeds. It's almost all coming from the U-joint, which I may try to shim or replace with one with tighter tolerances. Other than that, I need to do some suspension geometry tweaking to improve handling of lateral transients. Speaking of which, here's one last data capture. See if you can figure out what's going on here...</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYJF2s4iZZ23vq67hn4LED8aOPskF0KZovDts46zO3zVx295cuENferNij8TaXBNtQSesqqJPvndMxRkZJFxwKJXsmRsnv8cTG87SsOuJmGqoUu8M5jR-M5g92CaMolNT_0KhIhyphenhyphen5v_lk/s1799/tc02.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1799" data-original-width="1384" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYJF2s4iZZ23vq67hn4LED8aOPskF0KZovDts46zO3zVx295cuENferNij8TaXBNtQSesqqJPvndMxRkZJFxwKJXsmRsnv8cTG87SsOuJmGqoUu8M5jR-M5g92CaMolNT_0KhIhyphenhyphen5v_lk/w492-h640/tc02.png" width="492" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Mystery data log.</td></tr></tbody></table>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com2tag:blogger.com,1999:blog-8200098102909041178.post-48050271458694333842021-08-15T20:05:00.000-04:002021-09-20T11:33:23.190-04:00TinyCross: 4WD 80A Data Logging<p>It's been a long time since I did a proper test drive with TinyCross, although I've taken it out just for fun a few times. Since I completed the <a href="http://scolton.blogspot.com/2021/08/tinycross-weight-and-width-reduction.html">weight/width reduction pass</a> last week, I wanted to get it out again and do some proper data logging in 4WD, with the peak current set to 80A for all four motors. This is still below the ultimate target of 100-120A (for short bursts), but plenty for parking lot testing.</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh12KxTTTyG9aI5YuQI3DMFJxLUWkXY44dsriMHVLBHm3MpO8ht7ywQzf4PZmstQ2hCnNt-S6YKFTIF-rGs8OxMOljKY_K3on2Qgr3RoM3A6eQEnF_-s7eO6AVSOCPYCCYKVoLu9Dp8p_0/" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="2048" data-original-width="1215" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh12KxTTTyG9aI5YuQI3DMFJxLUWkXY44dsriMHVLBHm3MpO8ht7ywQzf4PZmstQ2hCnNt-S6YKFTIF-rGs8OxMOljKY_K3on2Qgr3RoM3A6eQEnF_-s7eO6AVSOCPYCCYKVoLu9Dp8p_0/w379-h640/tc93.jpg" width="379" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Really enjoying the extra 2" of clearance - I can get through most of the "doors" in my building now.</td></tr></tbody></table><p></p><p>I had to inflate the tires, but amazingly the air shocks don't seem to have leaked at all after a year of neglect. And they still do a pretty impressive job of soaking up the awful topography of my parking lot.</p><p><iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/XDjnYi42X1M" title="YouTube video player" width="640"></iframe></p><p>I wanted to do some more thorough data logging in 4WD to characterize some of the issues I've felt while just driving around for fun. The steering wheel PCB collects data from the front and rear motor drives over CAN, appends some of its own data, and writes the whole thing to a microSD card. When I first set this up, I just had it overwrite the existing data log every power cycle. But in the couple of years since I set that up, I've had to <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">master FatFs</a>. So setting it up to create new files on the fly without messing up any of the real-time stuff was an easy upgrade.</p><p>Here's what a 4x80A launch looks like:</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhn1AoiTrgVGp20NRQykuhhg999lD3uTm_jY7MSHTQ7IdnIjp6U1ggplaSq0j3KBjGTNY2LnWpcvAAcZDXdxGz35qYiHxNJQbFyb40acbHuepXU3z84tindKb53HViEqX4Baa5UCoIgkd4/" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="1939" data-original-width="1209" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhn1AoiTrgVGp20NRQykuhhg999lD3uTm_jY7MSHTQ7IdnIjp6U1ggplaSq0j3KBjGTNY2LnWpcvAAcZDXdxGz35qYiHxNJQbFyb40acbHuepXU3z84tindKb53HViEqX4Baa5UCoIgkd4/w399-h640/launch_4WD_80A_12V.png" width="399" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">4x80A launch (attempt).</td></tr></tbody></table><br />The main problem is pretty obvious from the data: the front wheels just don't have enough weight on them to support 80A. If there's even a little bit of a loose surface, one or both front wheels will lose grip. Excessive wheel slip is inefficient, so the peak acceleration isn't as high as it could be if all four wheels hugged their grip limit. But front wheel slip is especially bad because it results in massive torque steer. (I actually used this to make <a href="http://scolton.blogspot.com/2019/11/tinycross-4wd-and-servoless-rc-mode.html">remote-control TinyCross</a>.) It also has a habit of throwing rocks up into the driver's face.<p></p><p><iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/z502fdFy-yo" title="YouTube video player" width="640"></iframe></p><p>I've even debated whether the front wheel drive on TinyCross is worth the extra weight and complexity. <a href="http://scolton.blogspot.com/p/cap-kart.html#tinykart">tinyKart</a> handled pretty well with RWD only: I could put in a controlled amount of oversteer with the throttle. In fact, I got a chance to test out how TinyCross feels with RWD only when I had - let's call it an 80/20 failure - on the front right upright:</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYCRkEXaOWWCbNN61r3a1fcSchcpP1gHjec44NRyvGIW6-xFpKJ7jFwnWRcMg2PJslFACtz04h0N7OitiEYUgNkwKrvtI-yTn0S4uQB3V-H7jJuPbykC8cmmwqjfedKaduRzUZMcQaF6g/" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYCRkEXaOWWCbNN61r3a1fcSchcpP1gHjec44NRyvGIW6-xFpKJ7jFwnWRcMg2PJslFACtz04h0N7OitiEYUgNkwKrvtI-yTn0S4uQB3V-H7jJuPbykC8cmmwqjfedKaduRzUZMcQaF6g/w640-h480/tc94.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Always check your T-nuts! The only real casualty was the encoder wire.</td></tr></tbody></table><br />Although I was able to fix the mechanicals with the single hex driver I always bring with me, a few crimps pulled out of the encoder wire and I didn't have the tools to fix it. I could probably add a failover to sensorless operation for individual motors, but I'm not sure how well it'd work on the front motors, again because of torque steer. (Both fronts would have to agree to not produce torque until the flux estimator converges on the sensorless motor.) For now, I just removed power from the front drive.<p></p><p>In terms of handling, RWD works fine. But the launch is a mere 0.25g at 2x80A. There's no slip, and even if there was, it wouldn't matter as much on the rear since it doesn't induce torque steer.</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7JJ9sSoz9bSNl5rmhSiyUBPs08irLXoqaIDsJ4OwGvlGszha5OFHmyvF0PXw9ryL-kESON8gOjhoHwxB1kRlPp0rBygRP9MGS3Vt_7Zhzcpj9s5q6FG9Pl4hkXAP8sY0sSaibp-l8dsY/" style="margin-left: auto; margin-right: auto;"><img alt="" data-original-height="1929" data-original-width="1480" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7JJ9sSoz9bSNl5rmhSiyUBPs08irLXoqaIDsJ4OwGvlGszha5OFHmyvF0PXw9ryL-kESON8gOjhoHwxB1kRlPp0rBygRP9MGS3Vt_7Zhzcpj9s5q6FG9Pl4hkXAP8sY0sSaibp-l8dsY/w491-h640/launch_RWD_80A_12V.png" width="491" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">2x80A launch.</td></tr></tbody></table><br />Even at 120A, this would only be about a 0.4g launch. tinyKart, in its last and somewhat scary configuration, was hitting about 0.5-0.6g. Part of this is down to gearing: TinyCross, with 12.5" wheel, has to be geared for higher speeds. I could always ditch the front motors and switch to 80mm motors with more torque on the rear. But I think that goes against the spirit of TinyCross. Having full independent suspension and 4WD has always been the point.<p></p><p>So I think I'll finally have to dive in to writing some simple traction/launch control software. Just looking at the 4x80A launch data, it's easy to pick out the wheel that's slipping and imagine that the software could just fold back the current command to that wheel as its speed starts to diverge from the other three. But there are so many logical knots on the path to generalizing that to 4WD, where any subset of the four wheels could be slipping, that it makes my brain hurt to even think about.</p><p>There are some amazing technical <a href="https://www.tesla.com/blog/spin-stops-here">blog</a> <a href="https://www.tesla.com/blog/slip-sliding-away">posts</a> from the early days of Tesla (back when it was more of an engineering project than a consumer electronics device) where they talk about how it took months to go from a controller with excellent high-bandwidth torque control to functioning traction control, and even then a lot of it was subjective. One observation I really liked:</p><blockquote><p><span face=""Gotham Book", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif" style="background-color: black; color: #757575; font-size: 14px;"><i>This type of feedforward traction control can be hugely beneficial; for instance, it's much safer to avoid wheelspin altogether than react to it.</i></span></p></blockquote><p>This was regarding a lateral G observer that was fed into the friction model that the traction control software used to help limit motor torque to what it thought the tires could reasonably handle. This way, wheel slip might be limited to cases where there truly is a sudden drop in friction at one wheel. I think that should be the goal for this as well. I might even be able to just do slip detection on the front wheels. It'll be an interesting experiment, at least.</p><p></p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-56920304292855114682021-08-07T11:16:00.001-04:002021-08-07T11:16:15.270-04:00TinyCross Weight and Width Reduction Pass<p>It's summer, which means it's time to work on go-karts. This round, it's a modification to <a href="http://scolton.blogspot.com/p/cap-kart.html#tinycross">TinyCross</a> that I've been wanting to make ever since I <a href="http://scolton.blogspot.com/2019/09/tinycross-first-test-drive-and.html">first got it together</a> about two years ago. The main issue is that I designed it around stock <a href="https://electricscooterparts.com/electricscooterwheels.html#12-1/2%20Wheels">rear 12.5" scooter wheels</a>. These are almost symmetric and have threading on both sides of the hub that are meant for mounting the drive sprocket and brake disk. But - and this is maybe my favorite bit of packaging on this project - I've got the brake and drive sprocket both mounted to the inboard side, with the brake caliper sitting right in the middle of the belt:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjZInTH7Snqi9kuVbO6_2aCw1FDj0hdQTw27QfmUDP2N0jCCsm92FNtfFDJExwTdHKuDH8gsFsE19Jc5jOjDzROevc_UQLN0XrVk2JkD9zqpcCClyPZyaA13hammEYLOayzNpoh05sJag/s2048/tc47.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1650" data-original-width="2048" height="516" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjZInTH7Snqi9kuVbO6_2aCw1FDj0hdQTw27QfmUDP2N0jCCsm92FNtfFDJExwTdHKuDH8gsFsE19Jc5jOjDzROevc_UQLN0XrVk2JkD9zqpcCClyPZyaA13hammEYLOayzNpoh05sJag/w640-h516/tc47.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">The brake and drive sprocket are both mounted to the inboard side of the wheel, making the outboard side of the hub dead weight.</td></tr></tbody></table><p>This makes the extended length of the outboard side of the hub useless. But, I left it as stock for simplicity. I figured if I ever needed to replace the wheels, it would be easier to drop in a new stock 12.5" wheel. But, this drives the overall width of the kart up to about 35" for no good reason:</p><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkOQvFSUcoMrxGRvio06ZFFrPInsCfakz_ECXhtylDnF5QB4eMR4K-KHx0GzMbJXU9xSpqtpJVZssmz4OJOQYh5qY2mCBx97-uyoAS3LJYacUPRdBmw7hk2uV-kP8ZlOsbhu10dVUinW0/s1829/tc17.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1120" data-original-width="1829" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkOQvFSUcoMrxGRvio06ZFFrPInsCfakz_ECXhtylDnF5QB4eMR4K-KHx0GzMbJXU9xSpqtpJVZssmz4OJOQYh5qY2mCBx97-uyoAS3LJYacUPRdBmw7hk2uV-kP8ZlOsbhu10dVUinW0/w640-h392/tc17.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">The total width, about 35", is driven in part by the symmetric 12.5" wheel hubs.</td></tr></tbody></table><p></p><p>It's also unnecessary weight, especially factoring in the beefier 5"x5/8" hex standoffs I used to close the structural loop around each wheel. I figured I could eliminate 2" off the total width and about 1lb off the total weight if I just bit the bullet and re-machined the 12.5" wheel hubs. It still wouldn't fit through a 32" door frame, but it would be easier to wiggle through indoor spaces and fit in my car. It also would just look a lot nicer.</p><p>One of the reasons I put off this modification for so long is because I thought it would involve disassembling the entire wheel module, but it turns out that it's just barely possible to remove the wheel without removing the motor. I can take off the brake caliper and slip the belt off the pulley to give it just enough slack to pull the wheel off the spindle shaft. I don't remember intentionally designing it this way, but let's pretend I did. It'll be good for fixing flats, too. </p><p>The next obstacle to overcome was removing the outboard bearings. I didn't have a bearing puller on-hand, but I discovered that an 80/20 T-Nut (which I obviously have hundreds of...) is just about exactly the right size to push on the outer race of these bearings. So I came up with this improvised tool:</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBJMA1LHCMXpd3kuqDEqNUBvdijE3OCpYAU9rC59KhbOOT0G7CCD7uj_OLEiz6jLysv0EIHNT7-BGpp__CUPchHWh2I5b2UX3jO_zTKWRQiPPa61SqjDELx1UQBhSB4xnrMCX3kn0vkM8/s2048/tc83.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBJMA1LHCMXpd3kuqDEqNUBvdijE3OCpYAU9rC59KhbOOT0G7CCD7uj_OLEiz6jLysv0EIHNT7-BGpp__CUPchHWh2I5b2UX3jO_zTKWRQiPPa61SqjDELx1UQBhSB4xnrMCX3kn0vkM8/w640-h480/tc83.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Improvised bearing pusher.</td></tr></tbody></table><p>The tool is built <i>inside</i> the hub by slipping the 80/20 T-Nut through the bearing, flipping it horizontal, then dropping in the hex standoff from the other side. After fastening it together with a 1/4-20, it's ready for the press. Luckily, I didn't Loctite these bearings in, so they pressed out pretty easily.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjo_Sl81susGkM5UhT7P3L-BKn9hqEg1ycV15AT4O3kim2PfklSEOO4-9baHmBznM3h0aci1e5pLjQ6d56nxt-qmwPtcB20BINyAp9_QVLuhQztJfi32U42FJWbYuQ1oNMVUMm9UbLNc4/s2048/tc84.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjo_Sl81susGkM5UhT7P3L-BKn9hqEg1ycV15AT4O3kim2PfklSEOO4-9baHmBznM3h0aci1e5pLjQ6d56nxt-qmwPtcB20BINyAp9_QVLuhQztJfi32U42FJWbYuQ1oNMVUMm9UbLNc4/w640-h480/tc84.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Pressing out the bearings using the makeshift pusher.</td></tr></tbody></table><p>The 12.5" wheels don't fit on my <a href="https://littlemachineshop.com/products/product_view.php?ProductID=5100&category=1271799306">mini lathe</a>, but they do just barely fit on my <a href="https://littlemachineshop.com/products/product_view.php?ProductID=4190&category=1387807683">mini-mill</a>. I knew this ahead of time, so I bought a 22mm end mill specifically for cutting the new bearing pocket. (One of the nice features of this mini-mill is its use of a regular R8 spindle, so it's possible to get large tools for it.) I did have to get a little creative with fixturing. The brake disk is bolted down to a piece of 80/20, which is clamped in the mill. But, to make things stiff enough, I also had to ground the rim itself directly to the bed with some long clamping screws.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKicqFmiQEjtEHfYw8BnnQWUnH2TsUFmxCJwXJWIOvTIaq2TxQq973xHLyhxCoB_FiiOuFtlgmNshXtZ18qrs2ekQMzT7ibLEYZ4C4uSHf9Nuf_ZXlJ-AJu0rLD8JulMxNgSoscFVGG-U/s2048/tc88.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKicqFmiQEjtEHfYw8BnnQWUnH2TsUFmxCJwXJWIOvTIaq2TxQq973xHLyhxCoB_FiiOuFtlgmNshXtZ18qrs2ekQMzT7ibLEYZ4C4uSHf9Nuf_ZXlJ-AJu0rLD8JulMxNgSoscFVGG-U/w640-h480/tc88.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Clamping situation: not great, not terrible.</td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqea5kNOWyWUt8_ONoTngZ__FuDk6xFoLHkNSDlfCaJxnjhKCQ7LJeSMNFDBP3zmhdAJm5YaSzpPlG3acHmEMElhPiRtluuZYSjYOEGN-gTAFInTtYQ7Zf1rhyphenhyphen05ATWjY9bJ3zvbiqctU/s2048/tc89.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqea5kNOWyWUt8_ONoTngZ__FuDk6xFoLHkNSDlfCaJxnjhKCQ7LJeSMNFDBP3zmhdAJm5YaSzpPlG3acHmEMElhPiRtluuZYSjYOEGN-gTAFInTtYQ7Zf1rhyphenhyphen05ATWjY9bJ3zvbiqctU/w640-h480/tc89.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Pretty sure this mill was never meant to hold a tool this big.</td></tr></tbody></table><p>I decided to extend the bearing pocket by 1.000" first, before machining down the hub by 1.000". I'm not sure if this was the best order of operations, but it all went pretty smoothly. Here's 7:45 of relaxing slow-motion bearing pocket cutting, captured at 4K 420fps with my <a href="https://freeflysystems.com/wave">Wave</a>:</p><div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/F2xeFx_fLQE" title="YouTube video player" width="640"></iframe>
</div><p>These hubs are cast aluminum, so it wasn't surprising to find that there were some voids in the newly-machined faces. They're nothing that I think would affect the structural integrity, but it's an interesting consequence of the manufacturing process.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-NsEF-4nz8HD18iz9HfXtuh1tfSQcAbmzGbqOBUlyiH7HZOJX0mdMCBanmBpzeBWncU3X3BlZSrMfQCO2qEs6iBZi8bdVvRYynKQaPCn1ypWZT3wzFTTylpPr2nRRFGEvkhgYBT6p3FQ/s2048/tc90.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-NsEF-4nz8HD18iz9HfXtuh1tfSQcAbmzGbqOBUlyiH7HZOJX0mdMCBanmBpzeBWncU3X3BlZSrMfQCO2qEs6iBZi8bdVvRYynKQaPCn1ypWZT3wzFTTylpPr2nRRFGEvkhgYBT6p3FQ/w640-h480/tc90.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Casting voids exposed by re-machining the hubs.</td></tr></tbody></table><p>One of the downsides of doing this operation on the mill is that I didn't have a choice of machining the new bearing pocket to an interference fit. But I was pleased to see that, with all the extra effort put into stiffening the fixture, it was still a nice slip fit. I can always add Loctite later if needed.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0TSOSKcAGQj76WECEDPrstczHTz18_5pp48ZawBtc9SMWWZ1uD19e-en1sETtNvWG7RoL28jEGhlJAswPhokhnU92aVP8kf3ZZvptmvjfgRt5fMn2e6nCyT61RJ1b-fdrbDu97j_BSbY/s2048/tc91.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0TSOSKcAGQj76WECEDPrstczHTz18_5pp48ZawBtc9SMWWZ1uD19e-en1sETtNvWG7RoL28jEGhlJAswPhokhnU92aVP8kf3ZZvptmvjfgRt5fMn2e6nCyT61RJ1b-fdrbDu97j_BSbY/w640-h480/tc91.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">After re-machining, the bearings are now a nice slip fit.</td></tr></tbody></table><p>That just leaves the 7075 spindle shafts, which also needed to be shortened by 1.000". Cutting off the extra length and extending the outboard mounting hole was a quick task for the mini-lathe. Then, it just needed to be re-tapped.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-FBSVwMPVUvyY6Uk8Z6qGfRrPZ1FZX36MGJR1TyxVrJQKrkHoo4muhReCxPtK1VFU3RjTJxlvcPpHoq2sC3pxqvUOP4yV4Gyq16w6X1ULAJwrtDpZubONFcLxG3XevpUZOB5Ux-SQj7o/s2048/tc85.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-FBSVwMPVUvyY6Uk8Z6qGfRrPZ1FZX36MGJR1TyxVrJQKrkHoo4muhReCxPtK1VFU3RjTJxlvcPpHoq2sC3pxqvUOP4yV4Gyq16w6X1ULAJwrtDpZubONFcLxG3XevpUZOB5Ux-SQj7o/w640-h480/tc85.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Shortening the 7075 spindle shafts...</td></tr></tbody></table><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgA7SbJRIdyF_eVsb_no0dJQQaHrSmWWlW-zQpG7Hn_9zx5yXUkSvm29mVBg00fWr8pEkzCf_ziJGXA5ifP-3yvP3pJ32l35b1cgr-7EqbGv3BlSTsgISi0GM5FrLL-bupCBNJt5hyCrhU/s2048/tc86.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1536" data-original-width="2048" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgA7SbJRIdyF_eVsb_no0dJQQaHrSmWWlW-zQpG7Hn_9zx5yXUkSvm29mVBg00fWr8pEkzCf_ziJGXA5ifP-3yvP3pJ32l35b1cgr-7EqbGv3BlSTsgISi0GM5FrLL-bupCBNJt5hyCrhU/w640-h480/tc86.jpg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">...and re-tapping.</td></tr></tbody></table><p>Finally, I put everything back together, substituting much lighter 4"x1/2" hex standoffs to span the gap at the top of each wheel module. The total process took only about two hours per wheel, including disassembly and reassembly. So something I have put off for two years was really only one day of work...typical. Anyway, the final result is a kart that's now 2" narrower and about 1lb lighter.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4nm8JCOnDD-OLfM6tq5NrsUIkOIt3LMw3B6_3nE5WC6oIVkepVLy93RlRtuZvgl-S7xpAQCxUliOfArEhMGSC0fya5Gf0MNyBDAClBWpTGyi5ScesU7TwzJQPlBhbB_G0cLdJLjRJ09M/s2048/tc92.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="2048" data-original-width="1955" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4nm8JCOnDD-OLfM6tq5NrsUIkOIt3LMw3B6_3nE5WC6oIVkepVLy93RlRtuZvgl-S7xpAQCxUliOfArEhMGSC0fya5Gf0MNyBDAClBWpTGyi5ScesU7TwzJQPlBhbB_G0cLdJLjRJ09M/w610-h640/tc92.jpg" width="610" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">The pile at the front is roughly the weight saved. (5"x5/8" standoffs were replaced by 4"x1/2", but an equivalent amount of weight was taken out of each hub.)</td></tr></tbody></table><p>I have a few more tasks I want to do on this kart. It still needs to be fully weather-proofed. I have a plan for enclosing the motor drives, but need to figure out something for the steering wheel PCB. I may redesign that board from scratch since I don't think I'll ever get to using the battery balancing circuit on it. It can be much smaller and simpler without that. Lastly, there's always motor drive stuff to fiddle with to squeeze out more torque and/or speed.</p><p>For now, though, I'm glad it's a little lighter and a lot narrower. It'll make deploy that much easier, which ultimately means more actual testing and use.</p>Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-11412182198490830622020-04-18T16:44:00.000-04:002020-04-19T00:01:36.094-04:00Full-Speed CMV12000 Subsampled Readout: 1440fps 1080pNow that I've got a <a href="https://scolton.blogspot.com/2019/12/continuous-38gpxs-4k-400fps-image.html">continuous multi-Gpx/s image capture pipeline</a> running, it's time to rearrange some things to break the 1000fps barrier:<br />
<br />
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/DVDJMog8FxU" width="640"></iframe><br />
<br />
For this clip I'm using the CMV12000's X/Y subsampling mode to trade resolution for frame rate, hitting 1440fps at 2048x1088. The overall pixel rate is a little lower than in 4K (3.2Gpx/s vs. 3.8Gpx/s), so it's feasible to send this through the same Zynq Ultrascale+ capture pipeline, with some modifications, to record <span style="color: yellow;">continuously</span> to an NVMe SSD. With ~4:1 <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">wavelet compression</a>, this writes about 1GB/s to the drive, up to 1000s (16.7min) for a 1TB drive. That would be 16.7 <i>hours</i> of playback at 24fps, though. I figured 30 seconds real-time and 30 minutes of playback was enough water droplet footage for now.<br />
<h4>
CMV12000 Subsampling</h4>
In a <a href="https://scolton.blogspot.com/2019/12/continuous-38gpxs-4k-400fps-image.html">previous post</a>, I covered the pipeline architecture for continuously recording 400fps 4K video from a CMV12000 image sensor to an NVMe SSD. That was a 4096x2304 (16:9) frame, slightly larger than 4K UHD. The sensor's native resolution is 4096x3072 (4:3), which it can read in at 300fps. By reading in fewer rows, the maximum frame rate is increased. Going wider than 16:9 would allow frame rates higher than 400fps, but since the sensor always reads in full 4096px-wide rows, the speed gain is only linear.<br />
<div>
<br /></div>
<div>
To go much faster, it's necessary to read in fewer columns as well. Not all sensors can do this; reading whole rows may be baked into the hardware architecture. The CMV12000 doesn't support arbitrary readout width, but it <i>does</i> support 2x subsampling. In this mode, every other four-pixel square (<a href="https://en.wikipedia.org/wiki/Bayer_filter">Bayer</a> group) is skipped in both the X and Y directions. The remaining squares are transmitted on the LVDS channels using an alternate packing:<br />
<br /></div>
<div style="text-align: left;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUHR_JNG1oLfivmo_voboZejp7O5tye4u3WQU_3AjDujeNOz3C8spvC_V9zxxnKs9AygVPM7EHr7NUVM2OMyVT400Q4KaDbwr10Rs1qd5_ggSziNoXw2MKiMEVX0PSUPmHwtXoE3MyAxU/s1600/d31.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="571" data-original-width="1600" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUHR_JNG1oLfivmo_voboZejp7O5tye4u3WQU_3AjDujeNOz3C8spvC_V9zxxnKs9AygVPM7EHr7NUVM2OMyVT400Q4KaDbwr10Rs1qd5_ggSziNoXw2MKiMEVX0PSUPmHwtXoE3MyAxU/s640/d31.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">CMV12000 subsampled readout (color, X-flipped).</td></tr>
</tbody></table>
Each of the 64 LVDS channels alternates between two rows, with the lower 32 channels handling two even (G1/R1) rows and the upper 32 channels handling two odd (B1/G2) rows. This alternate data packing allows the subsampled image, with 1/4 as many total pixels, to be read out nearly 4x faster. There is a small amount of extra overhead time that makes the actual gain not quite 4x.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Subsampling drops the resolution from 4K to 2K but preserves the crop factor of the sensor, since the full width and height are still used. This is preferable to cropping a 2048px-wide image out of the middle. It doesn't give any increase in sensitivity though; to do that would require binning (averaging the larger 4x4 squares to generate the final 2x2). The CMV12000 does support binning, but the overhead is so bad that you might as well read out the 4K image and do it in post (assuming you have the data storage bandwidth, which I certainly do). So to go ~4x faster, I will need ~4x more light.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjapueHOJxckKJQkGpFka9rbmMEHa48pHPHydEPLfhhJwawwDDtR9RTlupYb6yJftKfFaToO6yQH3ml2CcKJiGaLwG_yD-2iZBXwdDrobl1koL7pKcwcYJum0xAVNDoNHxL6lBC8aQ2kk0/s1600/d33.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="461" data-original-width="1600" height="184" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjapueHOJxckKJQkGpFka9rbmMEHa48pHPHydEPLfhhJwawwDDtR9RTlupYb6yJftKfFaToO6yQH3ml2CcKJiGaLwG_yD-2iZBXwdDrobl1koL7pKcwcYJum0xAVNDoNHxL6lBC8aQ2kk0/s640/d33.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Light sensitivity of subsampling vs. binning.</td></tr>
</tbody></table>
</div>
<div style="text-align: left;">
Before worrying about a shortage of photons, though, I first need to deal with a shortage of programmable logic. To fit everything on the XCZU4, my main bottlenecks are BRAMs and LUTs. I managed to add the <a href="https://scolton.blogspot.com/2020/03/hdmi-hard-way.html">decoder for HDMI output</a> with no increase in either by sacrificing the third wavelet stage. But I've known for a long time that the day would come when I would need to add 128 more <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#h26">Stage 1 horizontal cores</a> to handle the subsampled inputs.<br />
<br />
It might seem odd that <i>more</i> cores are needed to process a smaller image. Even at the higher frame rate, the pixel input rate is lower than in 4K. Surely the existing horizontal cores could time-multiplex to handle the data? But, the wavelet cores must operate on groups of <span style="color: yellow;">adjacent</span> pixels. In this case, adjacency describes the nearest horizontal pixels of the same color, since applying a difference operation to pixels of different colors would not have the desired result. And whatever the color, pixels from another row are not horizontally adjacent. Since each LVDS channel now services two color fields <i>and</i> two rows, it must feed four independent wavelet cores.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVJVb8RhIu2_QCWmk24j_MC0vwdka419Ck2JwlA-pp0ZfR7JWjHK3O_8Mw5JhZara4gEZxVN3c4nZxL4Ujdo6yw8dY_dhcm6J2wUqv42c1yTreMCPLN7I96FZ7_P3K32pEdL9sTIFH1Pg/s1600/d34.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="604" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVJVb8RhIu2_QCWmk24j_MC0vwdka419Ck2JwlA-pp0ZfR7JWjHK3O_8Mw5JhZara4gEZxVN3c4nZxL4Ujdo6yw8dY_dhcm6J2wUqv42c1yTreMCPLN7I96FZ7_P3K32pEdL9sTIFH1Pg/s640/d34.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">In 2K mode, each LVDS channel feeds four independent Stage 1 horizontal cores.</td></tr>
</tbody></table>
So, the total number of Stage 1 horizontal cores doubles from 128 to 256. This jump has been on my mind since the early stages of the design, and I tried to optimize the horizontal cores as much as possible. A big part of this was reducing the operating pixel width from 16-bit to 12-bit, which brought the per-core LUT count down from 107 to 83. As this is the first stage of the pipeline, it's easy to verify that it won't saturate on 10-bit inputs. The horizontal cores operate in-line with the input using only distributed memory, so no additonal BRAMs are required. But there's now way around the additional 10,000 or so LUTs, and that will bring me right up to the limits of this chip.<br />
<br />
Since I knew there would be very few LUTs remaining for switching modes, I originally thought the 4K and 2K modes might have to exist as entirely separate PL configurations, their bitstreams loaded on as-needed by software. I've seen other cameras do this; it looks like a software reset when changing capture formats. And while it only takes a few seconds, I really dislike the workflow and the idea of maintaining two configurations.<br />
<br />
So, I spent some time looking at the actual differences between modes at all stages of the pipeline and decided that I could and should build the switch. I had this mode change in mind early in the design, so I tried to minimize the number of touch points required in each of the modules to switch between 4K and 2K. Even so, there are a number of small changes needed in the Wavelet, Encoder, and HDMI modules. They are collectively driven by a master switch in each module's AXI slave registers. I'll go through them in pipeline order below.<br />
<h4>
Wavelet Stage 4K/2K Switch</h4>
<div>
First, no actual switching is required to distribute the inputs to the Stage 1 horizontal cores; each channel always connects to the same four cores. Instead, the cores are gated by a master pixel counter based on their color and, when in 2K mode, also their row. The 2K mode switch turns on this extra enable gate and offsets the counter that handles first/last row states by one bit, to account for the half-width rows. Miraculously, this did not add any LUTs to the horizontal cores. I assume the extra logic just got merged into existing smaller LUTs...I'll take it.</div>
<div>
<br /></div>
<div>
The most complicated part of the switch happens next, at the interface between the Stage 1 horizontal and vertical cores. Instead of distributing outputs from four adjacent horizontal cores into a single row of a vertical core BRAM, the 2K interface distributes outputs from eight horizontal cores into two rows of a vertical core BRAM. Since the rows are half as wide, this takes the same number of pixel clock cycles (128). So, as will be the case at many points in the pipeline, this just boils down to rearranging the bits of the BRAM write address:<br />
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZmxTjD9PM8y5UBTWaXQvH4oKDgvd5GGoB-59vnduTnfUBx0WLJ4_mHK5TxP4cEmi5TA0rrp91obWI9iPQMMCoe_hUFo6La787x5RDkE0xs9fyKPVial6UoSXnIgc96CbrWJHPF_XkEBM/s1600/d35.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1431" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZmxTjD9PM8y5UBTWaXQvH4oKDgvd5GGoB-59vnduTnfUBx0WLJ4_mHK5TxP4cEmi5TA0rrp91obWI9iPQMMCoe_hUFo6La787x5RDkE0xs9fyKPVial6UoSXnIgc96CbrWJHPF_XkEBM/s640/d35.png" width="572" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Aspect ratio change and read/write addressing of the Stage 1 vertical core BRAMs in 4K vs. 2K mode.</td></tr>
</tbody></table>
Conceptually, the aspect ratio of the vertical core BRAM changes from 8 rows of 256px to 16 rows of 128px. The figure above shows where writes and reads occur in the BRAM at a given relative pixel count. Reads occur on half-counts since the Stage 1 vertical DWT operates at double px_clk frequency. The read address generator is also modified by the switch to account for the new aspect ratio. Only the eight most recent rows are actively written or read, so in 2K mode the BRAM is twice as big as it needs to be. The latency of the vertical core is also halved, since it's determined by the number of rows required to complete the vertical DWT operation. This will come into play later.</div>
<div>
<br /></div>
<div>
The Stage 1 vertical core buffers the alternating-row 2K mode inputs into a single-row format that's compatible with the rest of the pipeline, so changes after this point are relatively minor. Each Stage 1 vertical core feeds its output row to a Stage 2 horizontal core. The only modification required there is to offset the counter that handles first/last row states by one bit, to account for the half-width rows. Then, the Stage 2 vertical core just needs some more BRAM address rearrangement:<br />
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhUTb6GlFcFE0thLkQEPaYU3mwiUBVyIFG9BapvdltSUk4wJrdxviqu7uqHuasQyqxFHVCHLtfTDrhBRCiUdFcZMT5zMs3XqOFDXm7olEYbCDspAxsYaHf27SkfDiWCWGVPZeU8ZuS5gI/s1600/d36.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1430" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhUTb6GlFcFE0thLkQEPaYU3mwiUBVyIFG9BapvdltSUk4wJrdxviqu7uqHuasQyqxFHVCHLtfTDrhBRCiUdFcZMT5zMs3XqOFDXm7olEYbCDspAxsYaHf27SkfDiWCWGVPZeU8ZuS5gI/s640/d36.png" width="572" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Aspect ratio change and read/write addressing of the Stage 2 vertical core BRAMs in 4K vs. 2K mode.</td></tr>
</tbody></table>
</div>
<div>
Like the Stage 1 vertical core BRAM, the aspect ratio is changed from 8 rows of 256px to 16 rows of 128px. But since the first stage already rearranged things into single rows, the write addressing here is more straighforward: In both 4K and 2K mode, only a single row is filled at a time (by two adjacent Stage 2 horizontal cores). The row width is halved, but there's no write interleaving between the two rows. Ultimately, this is just a different arrangement of the write address bits. The read address generator is similarly modified to grab the right data for the Stage 2 vertical DWT. As with the Stage 1 vertical core, the BRAM is twice as big as it needs to be, and the latency is halved.<br />
<h4>
Encoder 4K/2K Switch</h4>
</div>
<div>
The compression stage doesn't care about the aspect ratio change, since the only context it uses for <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">variable-length encoding</a> is an immediate group of four pixels. However, it does need to know the adjusted latency of both wavelet stages, since the first pixel to be encoded will arrive sooner in 2K mode. For that, I just made all the latency offsets software-defined, through the encoder's AXI slave registers. And that should be the only change required here...</div>
<div>
<br /></div>
<div>
Except things are never that easy. I noticed after plugging in the expected latency values in 2K mode, two of the four color fields (R1 and G2) were actually dropping one pixel per row. It took a while to isolate this to the encoder, and then even more staring at this module to figure out what the problem was. Since the only change I made was to the latency offsets, I figured there had to be some fundamental difference between how the local pixel counter (px_count_e) drives the encoder states during row transitions with different offsets, and there was:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihI8urr1MktbJ59hVExn8RiBXKUJ2HyjISb_NZkSioQp4RO2OSqlMb-D9um3emvhW6A6Jb6fv4Z7Bnxbx6zWQxUO2aP6Z_7bF_Qyx3HaZddpzfDjx9ECM_FAIxO7Uteu7c4HmR2b1xGm0/s1600/d37.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="661" data-original-width="1600" height="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihI8urr1MktbJ59hVExn8RiBXKUJ2HyjISb_NZkSioQp4RO2OSqlMb-D9um3emvhW6A6Jb6fv4Z7Bnxbx6zWQxUO2aP6Z_7bF_Qyx3HaZddpzfDjx9ECM_FAIxO7Uteu7c4HmR2b1xGm0/s640/d37.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Encoder gating in 4K mode, showing the difference between sequential and combinational px_count_e_updated.</td></tr>
</tbody></table>
<div>
The above shows px_count_e at the first row overhead time (ROT) in 4K mode. It's negative since pixels haven't made it to the encoder yet, but the same behavior happens at all subsequent row transitions. During ROT, the sensor is not sending pixel data and all the pixel counters (including px_count_e) hold their previous values. A signal called px_count_e_updated is cleared, which gates the encoder from sending pixels to RAM (via an intermediate shift register called e_buffer). This signal was previously <span style="color: #bf9000;">sequential</span>, which would add one clock cycle delay between the ROT and when the encoder is gated. It should have been <span style="color: #76a5af;">combinational</span>, to line up correctly with the ROT.</div>
<div>
<br /></div>
<div>
But the write to e_buffer also only takes place every other group of four pixel clocks, for reasons discussed <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">here</a>. In 4K mode, the ROT happens to fall in a period where writes don't occur anyway. The <span style="color: #bf9000;">sequential</span> vs. <span style="color: #76a5af;">combinational</span> difference didn't matter to the final e_buffer_wr_en signal. But in 2K mode, the new latency offsets just happen to put the ROT one cycle before the start of a four-cycle write sequence, where the difference does matter:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq9d_tPpdApWawb4WQ_8rDniXIARHmWHPw9VoGfrBB_S7DOvJKlAFoe1WT1O-Q8JeM0G56R4Y24fCz2mMpHi7zRswXohsOw4-l7Jx3j7xPn2xerud5hfjRer1t_53hPnOgV2rm8kAXqME/s1600/d38.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="654" data-original-width="1600" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq9d_tPpdApWawb4WQ_8rDniXIARHmWHPw9VoGfrBB_S7DOvJKlAFoe1WT1O-Q8JeM0G56R4Y24fCz2mMpHi7zRswXohsOw4-l7Jx3j7xPn2xerud5hfjRer1t_53hPnOgV2rm8kAXqME/s640/d38.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Encoder gating in @K mode, showing the difference between sequential and combinational px_count_e_updated.</td></tr>
</tbody></table>
<div>
After switching over to <span style="color: #76a5af;">combinational</span> logic for px_count_e_updated, the missing pixel returned, and things were almost happy again. It turns out there was a similar issue at the quantizer and encoder modules themselves, before the write to e_buffer. This was simply due to them not being enable-gated at all, though. (Again, it must have been working thanks to lucky latency offsets in 4K mode.) Gating each with the same <span style="color: #76a5af;">combinational </span>px_count_e_updated signal worked fine.</div>
<h4>
HDMI 4K/2K Switch</h4>
<div>
But wait, isn't the <a href="https://scolton.blogspot.com/2020/03/hdmi-hard-way.html">HDMI output</a> always 1080p? While that is true, it doesn't mean there's nothing to be done here. In 4K mode, only the Stage 2 wavelet compression is decoded, leaving a 2K preview image (really, four color fields that are each 1024px wide) to be output via HDMI. This greatly reduces the size of the HDMI module, since it only has to decode four of the sixteen codestreams and do one stage of inverse DWT. However, getting to the same preview size in 2K mode would mean complete decoding, require all sixteen codestreams and two wavelet stages. I simply don't have room to do that, so I'm going to cheat.</div>
<div>
<br /></div>
<div>
The first step is to change how the viewport is mapped to a pixel count. To achieve <a href="https://scolton.blogspot.com/2020/03/hdmi-hard-way.html">arbitrary scaling of the preview image</a>, I first normalize the viewport to 16-bit, i.e. top-left (0, 0) to bottom-right (65535, 65535). The x and y components, vxNorm and vyNorm, are shifted around to create the pixel counters that drives the output pipeline. When switching from 4K to 2K, each component gets right-shifted by one and the split between x and y moves over by one bit in the final counter:</div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEkbR23hb4L8i0qTkHt8RbdVh6Aw-A2cUmI1yXffwigQYUnkUsOn9pWBFvdzN8Ug5XJ7R_fApDyQcRcIYBFBsKcg5ya5SvWk5QPmL880Opf14UWqC9Sb_rwEpdBG1E9dpnqU4pXZmsiPE/s1600/d39.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1494" data-original-width="1600" height="596" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEkbR23hb4L8i0qTkHt8RbdVh6Aw-A2cUmI1yXffwigQYUnkUsOn9pWBFvdzN8Ug5XJ7R_fApDyQcRcIYBFBsKcg5ya5SvWk5QPmL880Opf14UWqC9Sb_rwEpdBG1E9dpnqU4pXZmsiPE/s640/d39.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mapping between 16-bit (vxNorm, vyNorm) coordinates and opx_count in 4K vs. 2K mode.</td></tr>
</tbody></table>
This remapping means that the entire output pipeline operates at half resolution in 2K mode. The preview will actually just be scaled up from the four LL1 color fields, which are each 512px wide. There will still be bilinear interpolation to help smooth out the result, but it will be blurrier than the 1080p preview in 4K mode. But again there isn't really an alternative, at least not with the resources I have left on this chip.<br />
<br />
The output pixel counter (opx_count) drives all parts of the decoding process, starting with a RAM reading FIFO through the HDMI module's AXI master. No changes are required there or in the decoder itself, other than modifying the latency offsets accordingly. These have always been software-defined, so I just added the expected values for 2K mode and they worked without any hassle. (There was no equivalent sequential vs. computational bug, thankfully.)<br />
<br />
After this, the modifications to the Stage 2 inverse vertical wavelet cores are pretty simple and almost the same as in the forward direction. Each color field's IV2 core uses a single URAM for row storage. In 2K mode, the aspect ratio is changed from 16 rows of 1024px to 32 rows of 512px, by rearranging read and write address bits:<br />
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeQJMUR1VMC6IfHkE1mMJuX-3tVDYmPeQMMjB76hDmDY6XFdtD_Npm-diYnt9rRHHDW8s9IefcoZ5Ubcv60CLywzpcE7ngRH_R4-lVE-EYydISXl2ZUMN2EcLSDLW4y1ZV5OwU7AaTfms/s1600/d40.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1251" data-original-width="1600" height="500" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeQJMUR1VMC6IfHkE1mMJuX-3tVDYmPeQMMjB76hDmDY6XFdtD_Npm-diYnt9rRHHDW8s9IefcoZ5Ubcv60CLywzpcE7ngRH_R4-lVE-EYydISXl2ZUMN2EcLSDLW4y1ZV5OwU7AaTfms/s640/d40.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Aspect ratio change and read/write addressing of the Stage 2 inverse vertical core URAMs in 4K vs. 2K mode.</td></tr>
</tbody></table>
Unlike the forward direction, the Stage 2 inverse horizontal wavelet cores also use URAMs for row storage and these likewise need address bit rearrangement to change aspect ratios for 2K mode:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyu1eVYGwoVGI5k9opXNooIYAbLqd5pTCR-zbOacUsrdXqRDo0vgdBAG0MwEU__1BV9nKpFhzUSETi25rT0w2Ev4FV8e3f-dOp0dixxp4f1eQ_1NL401LkjZ2t3tjubsWHGyZUeKBwC4A/s1600/d41.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1143" data-original-width="1600" height="456" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyu1eVYGwoVGI5k9opXNooIYAbLqd5pTCR-zbOacUsrdXqRDo0vgdBAG0MwEU__1BV9nKpFhzUSETi25rT0w2Ev4FV8e3f-dOp0dixxp4f1eQ_1NL401LkjZ2t3tjubsWHGyZUeKBwC4A/s640/d41.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Aspect ratio change and read/write addressing of the Stage 2 inverse horizontal core URAMs in 4K vs. 2K mode.</td></tr>
</tbody></table>
And finally, the bilinear interpolation module needs to be adjusted to automatically scale up the preview image by 2x, so it can fill the viewport using the 512px-wide color field LL1 outputs. This can be done quickly by passing the shifted vxNorm and vyNorm values to the module, although this isn't <i>quite</i> correct, as will be discussed below. It's good enough for now, though.<br />
<h4>
Debayering</h4>
</div>
<div>
Applying an ordinary debayering algorithm, whatever it is, to the 2K subsampled raw data doesn't really work. This is because the physical spacing between pixels is no longer symmetric. For example, a red pixel is closer to its green and blue neighbors to the left and below than to the right and above. A proper bilinear interpolation needs to take this asymmetry into account, by modifying the location of pixel centers for each color field accordingly. More advanced algorithms are still built on the assumption of symmetric neighbors, so they'd all need modification to some degree.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFZSy6Xo_cGU0gVbQ_g4qYTTp36zpK9emQsNgJzjgqRHX28PuI9dNrzUGAi8V2-FtqOrZMhFbCvfKc2YcOGCT84KFq2sDbT_mLiyoMxY5AW7CVIYllu7eTSnRKNt_mZvWXQINBnsuzfdU/s1600/d42.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="735" data-original-width="1600" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFZSy6Xo_cGU0gVbQ_g4qYTTp36zpK9emQsNgJzjgqRHX28PuI9dNrzUGAi8V2-FtqOrZMhFbCvfKc2YcOGCT84KFq2sDbT_mLiyoMxY5AW7CVIYllu7eTSnRKNt_mZvWXQINBnsuzfdU/s640/d42.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Asymmetric neighboring pixels in subsampled mode can be handled by modifying interpolation pixel centers (left) or with an intermediate supersampling step (right).</td></tr>
</tbody></table>
<div>
Alternatively, the subsampled data can be supersampled by 2x to estimate the missing pixels (G2' and G1' in the image above) and then run through the ordinary debayer algorithm in 4K. The final output can then be scaled back to 2K to reflect the true information content of the data. This path takes longer for what may be an equivalent result for simpler debayer algorithms, but it might have advantages for more complex algorithms. All this will probably be obsoleted by neural networks that upscale 240p images to 16K in a few year anyway, so I'm not going to worry about it.</div>
<div>
<br /></div>
<div>
It is important to adapt the debayer algorithm for the subsampled pixel locations somehow, though, or there will be significant artifacts. The following comparison shows three different algorithms, nearest-neighbor, bilinear, and a <a href="https://www.microsoft.com/en-us/research/publication/high-quality-linear-interpolation-for-demosaicing-of-bayer-patterned-color-images/">Microsoft 5x5 interpolator</a> that I like. For each, a reference 4K capture and 4K debayer is compared to a 2K subsampled capture with an <i>unmodified</i> 2K debayer and a 2K subsampled capture with a supersampled 4K debayer.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBE7JHWM1fqJppgCMOwMIEAkvdhHZCQuVAQg9WCKkUPZAKNlqMas87BRK99I6pGKJEGU2vje312HSN-kVRQTpG3aCTVLVx5qmLxLydzkDEHlcieLgeA5L6J_1bk8dgKzjcSgKBM2PEfrw/s1600/d32.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1170" data-original-width="1600" height="468" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBE7JHWM1fqJppgCMOwMIEAkvdhHZCQuVAQg9WCKkUPZAKNlqMas87BRK99I6pGKJEGU2vje312HSN-kVRQTpG3aCTVLVx5qmLxLydzkDEHlcieLgeA5L6J_1bk8dgKzjcSgKBM2PEfrw/s640/d32.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Comparison of three different interpolation algorithms with 4K capture/debayer, 2K subsampled capture with unmodified 2K debayer, and 2K subsampled capture with supersampled 4K debayer.</td></tr>
</tbody></table>
<div>
None of these simple algorithms can do much to recover resolution - for that I defer to the AI supersampling state of the art - but using an unmodified 2K debayer on subsampled raw data creates significant color checkerboarding artifacts on edges. Supersampling the data by 2x and running a simple 4K debayer at least bypasses the problem of neighboring pixel asymmetry.</div>
<h4>
Resource Utilization</h4>
<div>
Squeezing in the 4K/2K switch was beyond what I'd hoped to fit on the XCZU4, but it just barely works. The switch itself really only adds LUTs where BRAM/URAM address bits are remapped or where pixel counts are shifted to account for the aspect ratio change. The main addition is the 128 new Stage 1 horizontal wavelet cores, which really push the resource utilization to the limits.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkNtnW8eXRBSrpisokxZZ8hljLMPVYElK24ls4CD6YbB2t1bKBvqvb1VJacjb_R0JFfB6fiSx1We3tYAfJFCGOIbzhOj9p3kHoyFA7ouGGQSuaqRIO1OInR8L-tWFrLoJjGRVs3lGvXgE/s1600/d43.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1189" data-original-width="1600" height="474" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkNtnW8eXRBSrpisokxZZ8hljLMPVYElK24ls4CD6YbB2t1bKBvqvb1VJacjb_R0JFfB6fiSx1We3tYAfJFCGOIbzhOj9p3kHoyFA7ouGGQSuaqRIO1OInR8L-tWFrLoJjGRVs3lGvXgE/s640/d43.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The XCZU4 with everything crammed in.</td></tr>
</tbody></table>
<div>
At this point I'm at 77143 LUTs (<span style="color: red;">87.82%</span>), 93883 FFs (53.44%), 118 BRAMs (<span style="color: red;">92.20%</span>), 14 URAMs (29.17%) and 146 DSPs (20.05%). But, since most of my cores are running at px_clk (60MHz) or HDMI clock (74.25MHz) frequency, the timing constraints are not too difficult to meet. The exception seems to be things that interact with the 250MHz AXI clock, including the encoder and decoder BRAM FIFOs. These have to have some amount of manual placement help to meet timing.</div>
<div>
<br /></div>
<div>
The good news is I don't really have much else to add to the programmable logic. I've already built in placeholder URAMs for UI overlays in the HDMI module, so those just need to be filled in by software. I might add some more color processing to the HDMI output, but that will mostly use DSPs, and possibly URAMs for color look-up tables, which should be no problem to add. I'm really happy that everything fits on the XCZU4, not just because the bigger chips are way more expensive, but because it's been a much better lesson in optimizing cores to fit resource constraints than if I had just switched to the XCZU7 early on.</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com3tag:blogger.com,1999:blog-8200098102909041178.post-66032159298419729402020-03-14T15:38:00.001-04:002020-03-15T16:39:23.606-04:00HDMI, the Hard Way<span style="font-weight: normal;">If I were to rank the components of this project in terms of the ratio of their actual vs. expected difficulty, the <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">NVMe interface</a> would probably be lowest, since it was nowhere near as hard as I thought it would be. The <a href="https://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">CMV12000 input</a> (easy, expected to be easy) and <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">wavelet engine</a> (hard, expected to be hard) would be somewhere in the middle. And the new top of the list, the hardest module that should have been easy, would be the HDMI output.</span><br />
<h4>
HDMI</h4>
There seem to be two main reference designs for outputting an HDMI signal from a Zynq SoC. Zynq-7000 series boards such as the <a href="https://www.xilinx.com/products/boards-and-kits/device-family/nav-zynq-7000.html">ZC70x</a> and <a href="http://zedboard.org/product/zedboard">Zedboard</a> use an external HDMI transmitter, the <a href="https://www.analog.com/en/products/adv7511.html">ADV7511</a>, to convert a parallel RGB interface into serial HDMI TMDS outputs. Zynq Ultrascale+ boards such as the <a href="https://www.xilinx.com/products/boards-and-kits/device-family/nav-zynq-ultrascale-mpsoc.html">ZCU10x</a> and <a href="http://zedboard.org/product/ultrazed-ev-carrier-card">UltraZed-EV Carrier Card</a> use the built-in serial transceivers of the ZU+ to drive the TMDS outputs through a <a href="http://www.ti.com/product/SN65DP159">SN65DP159</a> HDMI retimer. The latter is a more modern approach, supporting up to 4K60 through the <a href="https://www.xilinx.com/products/intellectual-property/hdmi.html">HDMI TX Subsystem IP</a>. But, that IP is not included with Vivado. It also requires three free GTH transceiver channels, which I don't have on the XCZU4. (Its four available channels are in use for <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">PCIe Gen3 to the SSD</a>.)<br />
<br />
There's nothing wrong with using an external HDMI transmitter with the ZU+, though. I left a PL GPIO bank open specifically for a parallel RGB pixel bus, either for an LCD controller or an HDMI interface. I opted for the slightly newer <a href="https://www.analog.com/en/products/adv7513.html">ADV7513</a>, which supports up to 1080p60 at 8-bit. This is perfectly acceptable as a preview and local playback resolution. Outputting a full 4K frame over HDMI might be useful for interfacing with a RAW recorder, but that is out of the question anyway at 400fps. In fact, I only really need a 24-30fps HDMI output, which means a very manageable <span style="color: yellow;">74.25MHz</span> pixel clock, based on the <a href="https://en.wikipedia.org/wiki/Extended_Display_Identification_Data#EIA/CEA-861_extension_block">CEA-861</a> standard.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3vpSdKZEnpJGF_LcxBw6Yqs-70ryYoD7HVV-XLGsFzkvh-IHk_bm2zQtxxfpL5MUFMhcuTkgZdzIKfW7uHqSyyvHPB1I4YMKyzDPYIVGZKD5seScXo816maUFc5QS-Gm8FXL3vGDpYlY/s1600/d15.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="876" data-original-width="1600" height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3vpSdKZEnpJGF_LcxBw6Yqs-70ryYoD7HVV-XLGsFzkvh-IHk_bm2zQtxxfpL5MUFMhcuTkgZdzIKfW7uHqSyyvHPB1I4YMKyzDPYIVGZKD5seScXo816maUFc5QS-Gm8FXL3vGDpYlY/s640/d15.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">HDMI timing parameters for 1920x1080p 24/25/30Hz with a 74.25MHz pixel clock.</td></tr>
</tbody></table>
Generating the required pixel clock, sync, and dummy RGB signals in the ZU+ Programmable Logic (PL) is pretty simple; I set that up as a module on day one of playing with HDMI. Typically, you'd just point this module at a frame buffered in RAM and let it pull the real data. (There are video DMAs and drivers that will do this more-or-less automatically.) But here's where I run into a <strike>slight</strike> problem: <span style="color: yellow;">I don't actually have a frame buffered in RAM.</span><br />
<h4>
The Hard Way</h4>
<div>
While it <i>is</i> possible to <a href="https://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">write the full 3.8Gpx/s raw frame data to RAM</a> on the ZU+, it would be futile to try doing any significant processing on it there. Even if I used all three 128b AXI bus connections between the PL and the memory controller at 250MHz, that would allow for less than three accesses per pixel...including the initial write. The Processing System (PS) has a similar memory access constraint, although processing pixels serially on the ARM cores is much too slow anyway. So I made the decision early on to implement the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">wavelet compression engine</a> in PL hardware and write the ~5:1 compressed codestreams to RAM instead, on their way to the SSD.</div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIq07duqPWJEDJEq6pYkHwZPG3eiPp6vT5ZGSH2Ur6qeCS4FFiS9K6mVe9zRPO52oDFPcpFqC08rdqbPbpUu-7aPBT99h2uJUwMoxE_Lp3ykM7YnED5Skr-7fY0N2rUD0SiMrM73MGTmM/s1600/d16.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="506" data-original-width="1600" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIq07duqPWJEDJEq6pYkHwZPG3eiPp6vT5ZGSH2Ur6qeCS4FFiS9K6mVe9zRPO52oDFPcpFqC08rdqbPbpUu-7aPBT99h2uJUwMoxE_Lp3ykM7YnED5Skr-7fY0N2rUD0SiMrM73MGTmM/s640/d16.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The capture pipeline, with the main data path highlighted and shown decreasing in width where compression occurs at the PL Encoder, before data is written to DDR4 RAM.</td></tr>
</tbody></table>
</div>
<div>
"No problem," you might say, "just split off raw data from the sensor and feed it to the HDMI module." Unfortunately, this doesn't quite work: In the time it takes the HDMI scan to complete one row, the capture pipeline has processed 50+ rows from the CMV12000. The input and output are just not in sync, and any attempt to buffer partial frames between them would require much more block RAM than I have available. It would also cause frame tearing that would ruin any attempt to preview periodic phenomenon with the global shutter.<br />
<br />
The only real choice is to put the HDMI output module after the RAM buffer, which means decoding compressed frame data on the way out:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRl8y6_7KMAkeDPjgVN6eajdY2Yth3EiNmhOSR76rrYxrZ9TiSdsjFc3Z0Q-yrPuWDnNVybvcHdeoGUn2AyZHcrLyCR7rMScYD8-6WnhwxCrrxY9c2D4U5SiSTgMjMFFQEYi1lN1_nCHw/s1600/d17.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="414" data-original-width="1600" height="165" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRl8y6_7KMAkeDPjgVN6eajdY2Yth3EiNmhOSR76rrYxrZ9TiSdsjFc3Z0Q-yrPuWDnNVybvcHdeoGUn2AyZHcrLyCR7rMScYD8-6WnhwxCrrxY9c2D4U5SiSTgMjMFFQEYi1lN1_nCHw/s640/d17.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The only logical place to put the HDMI output, and not just because I left space for it there in the block diagram.</td></tr>
</tbody></table>
The HDMI module reads codestream data from RAM as an AXI Master, decodes the pixel values, and runs an Inverse <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#dwt">Discrete Wavelet Transform</a> (IDWT) to recover the raw image. While this is a lot more work, it pays off twofold because the same module can be used for playback by reading frames back out of the SSD into RAM and pointing the decoder at them.<br />
<br />
Notwithstanding the design effort, the actual resource utilization of this module <i>should</i> be pretty low. For one, only four of the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">sixteen codestreams</a> need to be decoded to reconstruct a 2048px-wide image to use for the preview; there's no need to decode any LH1, HL1, or HH1 data. Also, the preview frame rate is at least 10x slower than the capture frame rate, so the amount of parallelism needed in the decoding and IDWT pipeline is much lower. Still, it's more logic on an already-crowded chip.<br />
<h4>
Kill Your Darlings</h4>
At this point I'm stubbornly committed to fitting this design on the XCZU4. With the capture pipeline complete, I was getting pretty close to <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikdQBoG5MfI1DHaQvQCQUamIizk6ojBcYeV_HQg25UOMJVD8jKQ1SHJyQLvcTmDhXsKYFGcKS-4ceZXHx6PKdp5HjMsP0KFZ5hA8meoNk3PZtrpX0nSE_QMIjEtsnmkj2JLmQnDQ7zuyk/s640/c90.png">maxing out this chip</a>, especially the LUTs (65593 / 87840) and BRAMs (122 / 128). And this was after a significant optimization pass on all the cores, including trimming pixel math operations from 16-bit to 12-bit where applicable and removing debug interfaces. These bottlenecks were already causing routing difficulty that was pushing up compile times, so I needed to make more room somehow. And then one day I woke up and decided to delete Wavelet Stage 3.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7XG1-7CGjFHmcQKD0ChNJMaQmH1JNOHYVo9n8fWkrehM8vRPspX7kwLppSwW4ijjQpHDWwc161mUH76E8gGzdNHNqE8hdXVlcIhjpo1KilPwNsrAiEpr577xNIsZm95W7fodREew8Td0/s1600/d18.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="741" data-original-width="1600" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7XG1-7CGjFHmcQKD0ChNJMaQmH1JNOHYVo9n8fWkrehM8vRPspX7kwLppSwW4ijjQpHDWwc161mUH76E8gGzdNHNqE8hdXVlcIhjpo1KilPwNsrAiEpr577xNIsZm95W7fodREew8Td0/s640/d18.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">An example showing the effect of deleting the third DWT stage without changing the target compression ratios of any other stages. The red bars are each sized proportionally to the compressed sub-band they represent.</td></tr>
</tbody></table>
<div style="text-align: left;">
Stage 3 only handles 1/16 of the total data throughput, but it is visually the most significant and thus uses the least amount of compression. In the example above, replacing Stage 3's output with a raw 1/4-scale average image (LL2) has a relatively small effect on the overall compression ratio. It's also not a complete loss, since the 1:1 LL2 will yield slightly better visual quality if the other subbands remain unchanged. The distribution of bandwidth that achieves the best image quality with an overall compression ratio of 5:1 is still an unknown, but ditching Stage 3 probably isn't restricting the search space too far.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Although Stage 3 is by far the smallest wavelet core, removing it also simplifies a lot of downstream logic. The "XX3" <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">encoder</a>, which previously handled all four Stage 3 subbands by cycling through different inputs and quantizer settings, now becomes a pass-through for raw LL2 data. It also now has the same latency as the HL2, LH2, and HH2 encoders. This latency is the new maximum and is significantly lower than the former XX3 latency. (It's no longer necessary to wait for six whole LL2 rows for the Stage 3 DWT.) There's a symmetric payoff on the decoder side as well.<br />
<br />
So while I'm sad to see it go, I think it's the right call for now. Having three stages probably does improve the compression performance (objectively, the PSNR at a given compression ratio), but I think I can still achieve good image quality at an overall ratio of 5:1 with only two. Not even including prospective decoder savings, the reduction in LUTs (-4575), FFs (-5320), and most crucially BRAMs (-8) is well worth-it.<br />
<h4>
Working Backwards</h4>
<div>
In may ways, the HDMI output module is just a mirror image of the pixel input pipeline, from the deserialized CMV12000 input pixels to the AXI Master that writes encoded data to RAM. The 74.25MHz HDMI clock runs a master pixel counter that scans across and down the output frame. Whereas the CMV12000 clocks in 64 pixels in parallel, though, the HDMI only has to clock out one.<br />
<br />
Or does it? Each HDMI pixel (in RGB 4:4:4 format) consists of an 8-bit red, green, and blue value, whereas the <a href="https://en.wikipedia.org/wiki/Bayer_filter">Bayer-masked</a> sensor input is split into four interleaved color fields. Each color field's decoded LL1 image will only be 1024px wide. One option would be to center this in the HDMI frame and pull the 8-bit R, G, and B values directly from each color field's LL1:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguPUXagsqYHBbevgavv8DZtocyl9iQSZofJ22SA7F_dCFkYrCn8GeqVlK7MKz78mh4u37AxLM_t2pk4RdL5p0EAtjeFsuaFGCL9idLxa42imQ7YI7b9LuKRP9N8HWFIigbxKoehJiCSOw/s1600/d20.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="737" data-original-width="1600" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguPUXagsqYHBbevgavv8DZtocyl9iQSZofJ22SA7F_dCFkYrCn8GeqVlK7MKz78mh4u37AxLM_t2pk4RdL5p0EAtjeFsuaFGCL9idLxa42imQ7YI7b9LuKRP9N8HWFIigbxKoehJiCSOw/s640/d20.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">1:1 scaling from LL1 color field pixels to HDMI pixels.</td></tr>
</tbody></table>
In this case, each HDMI clock requires one pixel from each of the four color fields (the two greens are averaged). The logic couldn't really get any simpler. But, it makes poor use of the 1920x1080 HDMI frame, especially for widescreen aspect ratios. An alternative would be to scale everything up by a factor of two:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijWbg9bHJ-8W7y4Bo9mBb-jwnKJGlXp7omfD3aoacQSZfVY3adZur_1cH96LQevvaxLD3w6UppadFAGP309OgKOeXLfTfaBI7hRjf4nYD3T5wE6iB2fAUhwiyABNqoXWrgANVaH8TlUjI/s1600/d21.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="766" data-original-width="1600" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijWbg9bHJ-8W7y4Bo9mBb-jwnKJGlXp7omfD3aoacQSZfVY3adZur_1cH96LQevvaxLD3w6UppadFAGP309OgKOeXLfTfaBI7hRjf4nYD3T5wE6iB2fAUhwiyABNqoXWrgANVaH8TlUjI/s640/d21.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">2:1 scaling from LL1 color field pixels to HDMI pixels.</td></tr>
</tbody></table>
Now, a debayering method has to be used to reconstruct the missing color values at each pixel. For this application, a simple average of the neighboring pixels would be fine. (The off-line decoder uses a more complex, higher-quality method.) Each HDMI pixel now references as many as four pixels from each color field. But, these pixels don't all update at each HDMI clock. The average pixel consumption from each color field is actually only one per four HDMI clocks, as expected from the 2:1 scaling factor.<br />
<br />
But a 2:1 scaled preview doesn't fit in 1920x1080. The cropping isn't too bad for widescreen aspect ratios, but it's unusable for 4:3. Switching between 1:1 and 2:1 scaling depending on the aspect ratio would work, but adds a lot of conditional logic for a still-compromised result. An arbitrary software-controlled scaling between 1:1 and 2:1 would be so much better. So, time to break out the DSPs:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmiGAsJxaY99ZwhL30mOi41m2xT0uULL7vEuRGiM276EljBIZ_LjgmfnHRWr4SuJPgL0lTcQvmLfzd8HNsljAoc5Uw87l7vOmTMZ0RIh4MF1_YkTCWyIYQ1Dh61v37gu9ZSku0SMlwe1o/s1600/d22.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="738" data-original-width="1600" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmiGAsJxaY99ZwhL30mOi41m2xT0uULL7vEuRGiM276EljBIZ_LjgmfnHRWr4SuJPgL0lTcQvmLfzd8HNsljAoc5Uw87l7vOmTMZ0RIh4MF1_YkTCWyIYQ1Dh61v37gu9ZSku0SMlwe1o/s640/d22.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Arbitrary scaling from LL1 color field pixels to HDMI pixels, using bilinear interpolation.</td></tr>
</tbody></table>
To achieve arbitrary scaling, the four 1024px-wide LL1 color fields are resampled onto a 65536px-wide grid, accounting for the offsets between the centers of pixels of each color. Then, a viewport is defined within the HDMI frame and normalized onto this 16-bit grid (<a href="https://github.com/coltonshane/WAVE-Vivado/blob/master/base_cmd.srcs/sources_1/ip/HDMI_1.0/src/viewport_normalize.v">using DSPs</a>). The four pixel centers of each color field that box in the normalized viewport coordinate are used for bilinear interpolation (<a href="https://github.com/coltonshane/WAVE-Vivado/blob/master/base_cmd.srcs/sources_1/ip/HDMI_1.0/src/bilinear_16b.v">using more DSPs</a>) to produce the R, G, and B values. This is also the debayer step, thanks to the pixel center offsets.<br />
<br />
One thing I actually do have plenty of is DSPs, and this seems like a great use for 14 of them. Being able to reposition and rescale the preview image from software makes life a lot easier. The down-side is that sixteen LL1 pixels are required to generate a single HDMI pixel. But as with the 2:1 case, the input pixels don't all change with every HDMI clock. The average LL1 pixel consumption rate will depend on the scale, but if the viewport width is always at least 1024px, <span style="color: yellow;">it will never exceed one LL1 pixel per color field per HDMI clock</span>. All upstream logic in the decoder is designed with this constraint in mind.</div>
<div>
<h4>
Ultra-IDWT</h4>
</div>
<div>
Next upstream is the Inverse Discrete Wavelet Transform (IDWT). One of the most significant simplifications achieved by deleting Wavelet Stage 3 is that the HDMI output module only has to do one stage of IDWT: Stage 2. This stage recovers LL1 from the LL2, LH2, HL2, and HH2 subbands. The order of operations is reversed in the IDWT: vertical first, then horizontal. Since we're working backwards from the HDMI output, let's look at the horizontal core first.</div>
<div>
<br /></div>
<div>
The <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#h26">forward horizontal DWT core</a> is heavily optimized for speed and size using only FF-based distributed memory. In the inverse direction, there's a lot more breathing room. Only four cores are needed (one per color field) and they only need to process at most one pixel per HDMI clock. So, I am able to combine the horizontal IDWT with a block RAM buffer and output shift register pretty easily. I'm almost completely out of BRAMs, but I have plenty of UltraRAM (URAM) for this.</div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVmwfD7l2pOyC-mZLKYt6IApt-eeXdPIpRNJVKgeQUVHa4FnzIrLeKKQEqB7ePHM2rRixn_3tx21P6k2yF6Ph1Q94Vkj8FM0WUSZyRhp0ThAjPZhGQtsuFgRD2NIneYtTc7JvssNQLzQ/s1600/d23.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="847" data-original-width="1600" height="338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVmwfD7l2pOyC-mZLKYt6IApt-eeXdPIpRNJVKgeQUVHa4FnzIrLeKKQEqB7ePHM2rRixn_3tx21P6k2yF6Ph1Q94Vkj8FM0WUSZyRhp0ThAjPZhGQtsuFgRD2NIneYtTc7JvssNQLzQ/s640/d23.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Horizontal IDWT and output buffer for one color field built around a single URAM.</td></tr>
</tbody></table>
Each URAM is 32KiB, enough to store 16 rows of LL1 data. The oldest two rows (N+0 and N+1) feed output shift registers that end in the four pixels the bilinear interpolator needs. The horizontal IDWT is performed on data from Row N+3, its result written back to Row N+2. As in the forward direction, pixels are processed in 64-bit groups of four: two interleaved pairs of low-pass and high-pass values become four LL1 outputs. Two half-speed shift registers unpack 64-bit URAM reads for the IDWT and pack the results into 64-bit writes. Running the IDWT as a single combinational step is not as efficient as using sequential <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#lifting">lifting steps</a>, as in the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#h26">forward horizontal DWT</a>, but it's a bit simpler to do with shift registers. Meanwhile, new data from the vertical stage is fed in at Row N+6.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZjif9ZBaLCqDZVjwxmTKBAWxvF4rA4d1jiF-lol9jalzBQtICYbuyG5JaG2YfJBRLR_lrYYLuQF9WiQBqZBeBvqbzgNmAdPLEcIc6Jltqk0IrWIHuW5ZuaUvL9O1wyJ-PScTmiVM8HYI/s1600/d24.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="706" data-original-width="1600" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZjif9ZBaLCqDZVjwxmTKBAWxvF4rA4d1jiF-lol9jalzBQtICYbuyG5JaG2YfJBRLR_lrYYLuQF9WiQBqZBeBvqbzgNmAdPLEcIc6Jltqk0IrWIHuW5ZuaUvL9O1wyJ-PScTmiVM8HYI/s640/d24.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Vertical IDWT for one color field built around a single URAM.</td></tr>
</tbody></table>
<div>
The vertical IDWT cores are also each built around a single URAM. In this case, the URAM is split in half for low-pass (HL2/LL2) and high-pass (HH2/LH2) vertical data. Four pixels each from three rows of low-pass data (N+0 to N+2) and one row of high-pass data (N+9) are processed every four clocks to create two four-pixel outputs to write to horizontal core URAM. In a shameful waste of clock cycles, input rows are scanned twice and the output write alternates between the even and odd IDWT results. (There are other ways to deal with the 2:1 row scanning ratio, but I'm willing to trade power for simpler logic right now.) Meanwhile, raw interleaved LL2, LH2, HL2, and HH2 data are written in to rows somewhere just ahead of the IDWT read pointers.<br />
<h4>
Decompressor and Distributor</h4>
</div>
<div>
Each horizontal and vertical core operates on a single color field, but the four input codestreams are instead separated by subband (LL2, LH2, HL2, HH2), with all four color fields being present in each codestream. The codestreams also cycle through four different column positions in a given row, since the Stage 2 <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#v26">forward vertical DWT</a> uses four cores in parallel. A distributor remaps decoded subband data to the appropriate write address in one of the vertical IDWT cores. This is also a good place to interleave the high-pass and low-pass data, which facilitates the horizontal IDWT.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSbVuaTRHKqYE4zE0ksESMelRSsWlTLKqSIBNYVURwYyLnBi_XyqY3yz99229RPmQ12Mw8S-KI24GfyxQIxfBv1vZNK51pXsoTlLAAVFAoTEI3s9atUmBGYNvNsxGkZKYg0i4pjo5-5Kk/s1600/d25.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="429" data-original-width="1600" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSbVuaTRHKqYE4zE0ksESMelRSsWlTLKqSIBNYVURwYyLnBi_XyqY3yz99229RPmQ12Mw8S-KI24GfyxQIxfBv1vZNK51pXsoTlLAAVFAoTEI3s9atUmBGYNvNsxGkZKYg0i4pjo5-5Kk/s640/d25.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">After decoding, subband pixels are redistributed to the appropriate location in each color field's vertical IDWT buffer.</td></tr>
</tbody></table>
<div>
The distributor writes four pixels into one of the four vertical core URAMs at most once per HDMI clock, to satisfy the one pixel per color field per clock constraint discussed above. For viewport widths greater than 1024px, the distribution is gated by the master pixel counter, which only updates when the interpolators actually need new pixels.</div>
<div>
<br /></div>
<div>
Continuing upstream, the distributor receives 16-bit signed pixel values from the four codestream decompressors. Each one takes in codestream data from RAM as-needed, decoding four pixels at a time by reversing the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">variable length code</a> used by the encoder. The pixels are then multiplied by the inverse of the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#quantizer">quantizer</a> multiplication factor, using more DSPs, to recover their full range.<br />
<br />
Raw codestream data is read in from RAM by an AXI Master into BRAM FIFOs at the entrance to each decompressor. I'm using precious BRAMs here, for the built-in FIFO functionality and to make the decoder RAM reader symmetric to the encoder RAM writer. A round-robin arbiter checks the FIFO levels to see when more data needs to be read. I'm only using a 64-bit AXI Master on the decoder, since the bandwidth already far exceeds the worst-case HDMI output requirement.<br />
<h4>
Start-Of-Frame Context</h4>
<span style="font-weight: 400;">So far, the HDMI output pipeline looks a lot like the sensor input pipeline in reverse. But one subtle way in which they differ is in Start-Of-Frame (SOF) context: the state of the pipeline at the beginning of each frame. In the interest of speed, the input pipeline is </span><i style="font-weight: 400;">not</i><span style="font-weight: 400;"> flushed between frames. Furthermore, codestream addresses for a given frame are updated during the Frame Overhead Time (FOT) interrupt, while some data is still in the pipeline, so the very bottom of Frame N-1 becomes the top of Frame N in memory.</span><br />
<span style="font-weight: 400;"><br /></span></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiszMho5Loj_K8ys4zmJTzo_FYvxHHJfofD_RV0-c_513wrL0mw7jKfRupBjW4fansiD9QOwZoRpUD79E-slqM5JcmwEqCU4FwmkjNmDZZwp-KOq5Zmpkhq1YfTKyGcJzoY2CKA7DdLr1U/s1600/d26.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="822" data-original-width="1600" height="328" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiszMho5Loj_K8ys4zmJTzo_FYvxHHJfofD_RV0-c_513wrL0mw7jKfRupBjW4fansiD9QOwZoRpUD79E-slqM5JcmwEqCU4FwmkjNmDZZwp-KOq5Zmpkhq1YfTKyGcJzoY2CKA7DdLr1U/s640/d26.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Overlap between Frame N-1 and Frame N in memory. SOF N marks the sector-aligned start of "Frame N" in RAM, set during the FOT interrupt from the CMV12000. The decoder seeks the actual start of Frame N data.</td></tr>
</tbody></table>
If the decoder processes every frame, this isn't a problem: it can wrap cleanly through the overlapping region to get the data it needs for both frames. But the HDMI output only processes a subset of the frames captured. It needs to be able to find the start of any individual frame and process it independently. This is needed for seeking in a playback context too. But I can't afford the time it would take to flush the input pipeline between each frame. So instead I need to completely capture the state of the pipeline at the SOF boundary.</div>
<div>
<br /></div>
<div>
As it turns out, this isn't too bad, since there are only a few places where data can remain in the input pipeline at the SOF: </div>
<div>
<ol>
<li>In the pre-encoder pixel memory: registers or BRAM buffers that are part of sensor input, DWT or quantizer operations. These have a fixed latency of <span style="color: yellow;">6336px for Stage 1+2</span>. The decoder can offset its pixel counter by this amount, essentially discarding the overlapping pixels into the space between VSYNC and the start of the viewport.<br /> </li>
<li>In the 128-bit <a href="https://github.com/coltonshane/WAVE-Vivado/blob/master/base_cmd.srcs/sources_1/ip/Encoder_1.0/src/compressor_16in.v#L94">e_buffer register</a> of each codestream that accumulates encoded data before writing it to that codesteram's BRAM FIFO. The number of bits remaining in this register is neatly captured by its write index, <span style="color: #9fc5e8;">e_buffer_idx</span>.<br /> </li>
<li>In the codestream BRAM FIFO itself. This is captured by the <span style="color: #b6d7a8;">FIFO read level</span>, already used as the AXI write trigger. Since these FIFOs are 64-bit write and 128-bit read, care must be taken to keep track the <span style="color: #d5a6bd;">write level LSB</span> as well, to know if there's an extra half-word in memory that can't be read yet.</li>
</ol>
<div>
The last two combine to give a number of bits to discard for each codestream: </div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b><span style="color: #9fc5e8;">e_buffer_idx</span> + 128 * <span style="color: #b6d7a8;">fifo_rd_count</span> + 64 * <span style="color: #d5a6bd;">fifo_wr_count[0]</span> </b></span></div>
<div>
<br /></div>
</div>
<div>
To fully capture the SOF context, these three values are written to the frame headers during the FOT interrupt. A VSYNC interrupt from the HDMI module prompts software to read the header of the next frame to be displayed, calculate the number of bits to discard for each codestream, and pass it to the decoder along with the codestream start addresses. That number of bits are then discarded by the decoders prior to attempting to decode any pixels.<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_rQI5u2AfhMstpVqom6F5hR8N6R7H_4PDYG-BES-lCd_dzWwmvPMDQkLKb6HjL5413bKQ7LaXLVFsvL7r8SnXuuggcu6b-fJQwqqCSjk7K9ij7IpmiRRjjrajXD_0hcFig3JSW3w4RGA/s1600/d27.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="602" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_rQI5u2AfhMstpVqom6F5hR8N6R7H_4PDYG-BES-lCd_dzWwmvPMDQkLKb6HjL5413bKQ7LaXLVFsvL7r8SnXuuggcu6b-fJQwqqCSjk7K9ij7IpmiRRjjrajXD_0hcFig3JSW3w4RGA/s640/d27.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">High-level architecture of the encoder and decoder interactions with the CPU and RAM.</td></tr>
</tbody></table>
<div>
In total, the HDMI output module (decoder and all) uses 4363 LUTs, 4227 FFs, and 4 BRAMs, less than what was saved by deleting Wavelet Stage 3. It adds 8 URAMs and 26 DSPs, but I'm not running short of those (yet). Except for the AXI Master, it runs entirely on the 74.25MHz HDMI clock, so it shouldn't be too much of a timing burden. There might be room for a bit more optimization, but I'm happy with the functionality it gives for its size.</div>
<h4>
Focus Assist</h4>
<div>
<div class="separator" style="clear: both; text-align: left;">
The main reason I wanted to get the HDMI module done now, ahead of some of the other remaining tasks, is so I can use the real-time preview for testing. It sucks to have to pull frames off one-by-one through USB to iterate on framing, exposure, and especially focus. Having a 1080p 30fps preview on something like an <a href="https://www.atomos.com/shinobi">Atomos Shinobi</a> on-camera monitor makes life a lot easier, and moves in the direction of standalone operation.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/uceEuX6Fj1Q" width="640"></iframe>
</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
One neat trick you an do with wavelets is overmultiply the high-pass subbands (LH1, HL1, HH1) to highlight edges in the preview image. This effect is useful for focus assist. Most on-camera monitors can do this anyway (by running a high-pass filter on the HDMI data), but it's essentially free to do in the decoder since the subbands pass through a multiplier anyway to undo the quantization division. I'll take free features any day.</div>
<h4 style="clear: both; text-align: left;">
Macro Machining</h4>
</div>
<div>
With the newfound ability to actually focus the image in a reasonable amount of time, I'm finally able to play with a new lens: The <a href="https://www.irixusa.com/irix-cine-150mm-t3-0-macro-1-1">Irix Cine 150mm T3.0 Macro</a>. I started drooling over this lens for close-up high-speed stuff after watching <a href="https://www.youtube.com/watch?v=GsxybNZn22k">this review</a>. I'm no lens expert, but I feel like this lens series competes with ones 3x its price in terms of image quality. My first test was to attempt to get some macro shots of my mini mill:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_AkJ_UoOP1WlnCvyze6txyfz3e7AiGOB0IO6q1lsn_T00jRvUCeVZ70lVbfuNJUwyuoR3pqGpfyTS3C9QOlP8aJn-WSm_H8yWM2EJGLtphh3QKEDLzjk73r2foDrzzFCkZQfNlqP0Eow/s1600/d29.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_AkJ_UoOP1WlnCvyze6txyfz3e7AiGOB0IO6q1lsn_T00jRvUCeVZ70lVbfuNJUwyuoR3pqGpfyTS3C9QOlP8aJn-WSm_H8yWM2EJGLtphh3QKEDLzjk73r2foDrzzFCkZQfNlqP0Eow/s640/d29.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Shooting my mini-mill at 400fps with the Irix 150mm T3.0 Macro lens.</td></tr>
</tbody></table>
<div>
The HDMI output was crucial for this, since the lens has an insanely shallow depth-of-field at T3.0, less than the width of the cutting tool. The CMV12000 is not a particularly good low-light sensor, so with an exposure time of around 1.87ms, I needed to add a good deal of light. To make things more interesting, I threw in some cheap <a href="https://www.ikea.com/us/en/p/dioder-led-4-piece-light-strip-set-multicolor-50192365/">IKEA RGBs</a> as well. It took a while to get set up, but the result was promising:<br />
<br /></div>
<div>
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/qQ8LboNODIs" width="640"></iframe></div>
<div>
<br />
I'll probably repeat this with a more interesting subject (this was just a piece of scrap aluminum) and a more stable mount. If I can get more light, it might be good to close down to T5.6 or so as well, to get a bit more depth of field, and drop the exposure to 180º shutter for less motion blur on the cutter. But the lens is terrific and I'm happy with the quality of the two-stage wavelet compression so far. The above clip has an average compression ratio of right around 6:1, helped along by the ultra-shallow depth of field.<br />
<h4>
Next Up</h4>
</div>
<div>
The last major HDL task on this project is modifying the pipeline to accept 2K subsampled frames from the sensor at higher frame rates (up to around 1400fps at 1080p!). This will probably be a separate Vivado project and bitstream, since it requires substantial modifications to the input pipeline. It also needs twice as many Stage 1 horizontal cores, since four rows are being read in simultaneously instead of two.</div>
<div>
<br /></div>
<div>
But I may tackle some simpler but no less important usability tasks first. For one, I still don't have pass-through Mass Storage Device access to the SSD over USB C. This is necessary for getting footage off without opening the camera (or using RAM as intermediate storage). With that and a bit of on-camera UI work (record button, simple menus), I can finally run everything completely standalone soon.</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com3tag:blogger.com,1999:blog-8200098102909041178.post-43908893068349727892019-12-19T00:46:00.002-05:002019-12-26T13:35:33.598-05:00Continuous 3.8Gpx/s (4K 400fps+) Image Capture Pipeline<div>
In the original <a href="https://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">Freight Train of Pixels</a> post, I laid out three main technical challenges to building a <i>continuous recording </i>3.8Gpx/s imager. All three have now been dealt with, using a Zynq Ultrascale+ SoC as a hardware base. The detailed implementations for each one has its own post:</div>
<div>
<br /></div>
[<a href="https://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">The Source</a>] - Full-speed read-in of the CMV12000's 64 LVDS channels.<br />
<div>
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">The Pipe</a>] - Hardware wavelet compression engine.</div>
<div>
[<a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">The Sink</a>] - Sustained 1GB/s writing to an NVMe SSD.</div>
<div>
<br /></div>
<div>
Now it's time to put all three pieces together and run it as a full pipeline:</div>
<div>
<br /></div>
<div>
<div style="text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/j_u-m2xwpek" width="640"></iframe></div>
<div style="text-align: center;">
<span style="font-size: x-small;">Since YouTube uploads are at the mercy of H.264, here's a <a href="https://scolton-www.s3.amazonaws.com/video/wv37.png">PNG frame</a> as well.</span></div>
<br />
There are lots of technical details to dive into, but the first thing to point out is that this is 12000 frames of continuously-recorded 4K 400fps video. That's 30s in real-time and 500s of playback at 24fps, something very few existing high-speed imaging systems can do. And I can keep going. This clip is "only" 24GB of a 1TB SSD. To fill the entire 1TB would take about 20 minutes at this bit rate. That's 20 minutes of real-time, 5.5 hours of playback at 24fps.</div>
<div>
<br /></div>
<div>
This is made possible mostly due to the insane speed of modern SSDs. High-speed cameras typically use RAM to buffer raw frame data in real-time, transferring short clips to non-volatile storage after the capture period. But with NVMe flash write speeds now well into the GB/s range, maybe continuous direct recording to a single drive (or a RAID 0 array) will catch on. Besides the ability to capture long clips, it also allows an alternative trigger-free user interface that would be familiar to anyone: push to record, push to stop.</div>
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie6TC8wYZVLs7hcT_VLO219K5i9tB5_wa6dmB7ADx7ObOD-0Jgb9FEkcOBLR89XvEAbxnKs65GerU48aoV99JfYAqTFcot18W1uSsBv-6pqeMG2AQj6Iv328hIMcpOrX5Oih4nPY3NK8c/s1600/d02.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="826" data-original-width="1600" height="330" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie6TC8wYZVLs7hcT_VLO219K5i9tB5_wa6dmB7ADx7ObOD-0Jgb9FEkcOBLR89XvEAbxnKs65GerU48aoV99JfYAqTFcot18W1uSsBv-6pqeMG2AQj6Iv328hIMcpOrX5Oih4nPY3NK8c/s640/d02.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The real MVP here is the 1TB Samsung 970 Evo Plus NVMe SSD, a good example of modern consumer electronics running laps around everything above it in the pro/industrial world.</td></tr>
</tbody></table>
Of course, the other enabling factor is the use of <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">wavelet compression</a> to reduce the data rate by a ratio of at least 5:1. This might seem like cheating, but since a similar compression ratio is utilized by <a href="https://www.red.com/red-101/redcode-file-format">REDCODE</a>, <a href="https://www.blackmagicdesign.com/products/blackmagicraw">Blackmagic RAW</a>, <a href="https://support.apple.com/en-us/HT208671">Apple ProRes RAW</a>, and probably many other "raw" formats, I don't feel the least bit guilty. On a sensor like the CMV12000, a lightweight compression pass might actually <i>help</i> the image quality, since it'll inherently denoise the image somewhat.</div>
<div>
<br />
I also picked a good hardware platform for this project: the lower-end Zynq Ultrascale+ SoCs, like that of the wonderful <a href="https://shop.trenz-electronic.de/en/TE0803-03-4AE11-A-MPSoC-Module-with-Xilinx-Zynq-UltraScale-ZU4CG-1E-2-GByte-DDR4-5.2-x-7.6-cm?c=452">XCZU4CG module</a> I've been using, have just barely enough resources to pull it off. I'm using 134 of the 144 LVDS-capable I/O, all four PCIe Gen3-capable transceivers, and most of the programmable logic. I was actually going to move up to the XCZU6, which has more than double the logic, but it seems to be on a different branch of the product tree that a) doesn't have PCIe hard blocks and b) isn't supported by <a href="https://www.xilinx.com/products/design-tools/vivado/vivado-webpack.html">Vivado HL WebPACK</a>. For now, I'll just try my best to optimize for the XCZU4. I still have the XCZU5 and XCZU7 available to me if needed, though.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-euC3DPK7TDJVDMQeAXwRbd6BwWAVMxbmd5W2Ca5D7gXEdE7cIMUyFAkl_Bw-_HlGI8t8CHET1nauDpPjeGS4zsLsyQIncKm_TUyYwWkkO3bXzsJKZmn8QwW1YiMj88ZsLMDMHCtqA-8/s1600/d03.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-euC3DPK7TDJVDMQeAXwRbd6BwWAVMxbmd5W2Ca5D7gXEdE7cIMUyFAkl_Bw-_HlGI8t8CHET1nauDpPjeGS4zsLsyQIncKm_TUyYwWkkO3bXzsJKZmn8QwW1YiMj88ZsLMDMHCtqA-8/s640/d03.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">XCZU4 module transferred over to the v0.2 carrier, which has some new connectors including a barrel jack for power in, a full-size HDMI out, and some isolated GPIO.</td></tr>
</tbody></table>
Although I'm sticking with the same ZU+ module, I did finally get around to building a <a href="http://scolton.blogspot.com/2019/11/zynq-ultrascale-superspeed-ram-dumping.html">new carrier</a>. The main addition is an HDMI transmitter, which I'll probably play with in the coming weeks. There's also a new barrel jack power input and an isolated GPIO connector. Other than a PCIe reference clock routing fix, I didn't touch any of the existing high-speed signals. I also (re)discovered good drag soldering technique, so there were zero issues with the 160-pin headers this time around.<br />
<br />
Most importantly, I felt confident enough in the design now to risk a color sensor on this board. Unlike my stockpile of monochrome CMV12000s from eBay, I bought this one new, at full price, and I do not want to break it. I haven't had any power supply issues in months of testing on the v0.1 board, so after a thorough multimeter check on the v0.2 board, I permanently soldered on the color sensor and crossed my fingers.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiC88Wjh6bDo85EkZqPvgmzbxLt0t0CiI8Dwuv7OMpfye8v7xtJP3UxLR3yIUfav5uiFn0NGwbgnGlwMmvq2HtynmOsiu-YDo91KnxjQZAOHSOudhrxbQtBZHoL8oo-LtmFESg0r369Gow/s1600/d01.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1015" data-original-width="1600" height="406" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiC88Wjh6bDo85EkZqPvgmzbxLt0t0CiI8Dwuv7OMpfye8v7xtJP3UxLR3yIUfav5uiFn0NGwbgnGlwMmvq2HtynmOsiu-YDo91KnxjQZAOHSOudhrxbQtBZHoL8oo-LtmFESg0r369Gow/s640/d01.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My one and only CMV12000-2E5C1PA, now committed to this board.</td></tr>
</tbody></table>
The wavelet compression engine was designed for the <a href="https://en.wikipedia.org/wiki/Bayer_filter">Bayer-masked</a> color sensor, with independent encoding for each color field, so I didn't have to change any hardware or software to accommodate it. All the color processing happens off-board, a point of reckoning discussed below. But from an image capture point of view, it's 100% drop-in.<br />
<br />
Returning to the integration of the three main pieces of the image capture pipeline, there were a handful of small but important details to sort out regarding how frames are buffered in RAM and how they are written out to files on the SSD:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR35A5fLNpd6vji6Qe1LZ9pbJJcUoHgCCbGWokiGqTLrwsw8bbzEjw6dlV8bGpGzN0_nUXCSBSXv34QU_TL4f2gxVSFymLP-ZgjKhDvVA36d8YnikLo0igtFC2EK8ZXclg_eHH_jF-x4s/s1600/d04.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="858" data-original-width="1600" height="342" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR35A5fLNpd6vji6Qe1LZ9pbJJcUoHgCCbGWokiGqTLrwsw8bbzEjw6dlV8bGpGzN0_nUXCSBSXv34QU_TL4f2gxVSFymLP-ZgjKhDvVA36d8YnikLo0igtFC2EK8ZXclg_eHH_jF-x4s/s640/d04.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
<ol>
<li>The AXI Master port on the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">Encoder</a> module now transfers data from the 16 compressor codestream FIFOs to RAM in increments of 512B, one logical block / sector on disk. This greatly simplifies downstream file writing operations.<br /> </li>
<li>The (now sector-aligned) addresses of the next RAM write for each compressor codestream are presented to software via the Encoder module's AXI Slave. These addresses increment automatically as data is written, but can be overwritten by software.<br /> </li>
<li>The <a href="https://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">CMV Input</a> module generates an interrupt at the start of the Frame Overhead Time (FOT), a short (~20-30μs) period of deadtime between frame readouts where there shouldn't be any RAM writing happening.<br /> </li>
<li>During the FOT interrupt, software reads the Encoder RAM write addresses, resets them if necessary, and records the start address and size of each compressor codestream for a given frame as part of a 512B frame header. Frame headers are stored in a circular buffer in RAM.<br /> </li>
<li>In the main loop, frames are dequeued from RAM as needed by writing the frame header, followed by each of the 16 codestreams it references, to a file with FatFs. A new file is created every so often to prevent the file size from exceeding the 4GiB FAT32 limit.</li>
</ol>
<div>
This is all pretty easy to implement, at least compared to the three main hardware modules themselves. Most of it is ARM software, which can be iterated and debugged much faster than programmable logic. Also, having the codestream RAM address and size baked into the frame header helps with validating and analyzing the output:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0TuntkfrLI6elCBDrA24wU-85RdsP0VkREmY0lsg-0Bs_BTV9VYQN9L48NLVsGAp7s65MWueG6wVfWsay3smu6bRN1UUYJwI4ve6BzSPzq-g1I4z8gMARXQ-Vv9qanjVnihj0QXmSH6w/s1600/d05.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="913" data-original-width="1600" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0TuntkfrLI6elCBDrA24wU-85RdsP0VkREmY0lsg-0Bs_BTV9VYQN9L48NLVsGAp7s65MWueG6wVfWsay3smu6bRN1UUYJwI4ve6BzSPzq-g1I4z8gMARXQ-Vv9qanjVnihj0QXmSH6w/s640/d05.png" width="640" /></a></div>
</div>
<div>
The codestream size per frame shows how bandwidth is being distributed to the 16 codestreams, with more bits per frame going to the low-frequency subbands. Compression ratios relative to raw 10-bit data are shown on the codestream size axis. Spikes can be seen during the portions of the clip where the steel wool burns brightest. There's also a gradual increase in codestream size over the 30 seconds, especially in the high-frequency subbands, which I believe is due to image sensor noise increasing with temperature. Feeding the codestream size back into the <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#quantizer">quantizer</a> settings to maintain a roughly constant bit rate will be a problem for another day.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi845mjezk9vrixjzFb_BslHpS5GJfpiDXW47F-fz23sFKHHZT1qZWEpb_7YtloNvvdJ7wdWopP9FeZCZ5GY5KXflspCFMbbd7OUw_AjvtxxIvETTdMtLTOm-YfgdGDTCRUCDW0g3U5KPQ/s1600/d06.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1061" data-original-width="1600" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi845mjezk9vrixjzFb_BslHpS5GJfpiDXW47F-fz23sFKHHZT1qZWEpb_7YtloNvvdJ7wdWopP9FeZCZ5GY5KXflspCFMbbd7OUw_AjvtxxIvETTdMtLTOm-YfgdGDTCRUCDW0g3U5KPQ/s640/d06.png" width="640" /></a></div>
Each codestream is given a range of RAM addresses to use as a circular buffer for frame data. At the start of a clip, the encoder RAM write addresses are set to the bottom of their range. As data is written, the addresses increment automatically. When they reach the top of their range, software resets them during the FOT interrupt. Each codestream does this independently, but the overall frame buffering capability is limited by the most frequently reset stream. In this case, XX3 resets approximately every 400 frames, so a maximum of 1s can be buffered. Ideally, the RAM ranges would be sized proportionally to the codestream bit rates to maximize the number of frames that can be buffered.</div>
<div>
<br /></div>
<div>
The RAM frame buffer allows the pipeline to ride out NVMe writing delays that last for several frame intervals. Each frame is time stamped once when it is read into RAM and again when it is submitted for writing to the SSD. The two time stamps and the frame write backlog count are added to the frame header, and can be used to review the delay.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9N_CN69qJsa4yPlpgOkR6ZMlZ8nVFhWBoKiW_DBxY0YWJqA4d6cfr7n4STxQI3a_nU-JI7cZhz2qjtgJZVlM23y2TAZfYJTTOp13xFwA0eElbgFMYCfMGw990ollo2Z1FIY6RwFEJCjU/s1600/d07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1224" data-original-width="1600" height="488" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9N_CN69qJsa4yPlpgOkR6ZMlZ8nVFhWBoKiW_DBxY0YWJqA4d6cfr7n4STxQI3a_nU-JI7cZhz2qjtgJZVlM23y2TAZfYJTTOp13xFwA0eElbgFMYCfMGw990ollo2Z1FIY6RwFEJCjU/s640/d07.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
Here, the read-to-read interval between frames is a steady 2.5ms, as it must be for 400fps. (A spike would indicate a dropped frame.) The read-to-write delay is typically 10ms, corresponding to a four frame backlog. This offset is set by software to allow for a small number of frames between the read and write pointer, which could be used to generate a low-latency local preview. There is a spike in read-to-write delay every 240 frames when a new file is created, since the file creation NVMe operations are <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-fatfs-with-bare-metal.html">slower than streaming writes</a>. Most of these spikes are only one frame, but there were instances of higher delays, up to 20 total frames (50ms). This is still easily absorbed by the RAM buffer, although it would be good to understand where the extra delay comes from.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
So all 12000 frames (113.2Gpx) made it onto the SSD in 30 seconds. That's the last stop on the freight train of pixels - once they hit the flash, they're no longer volatile or time-critical. So in some sense this project is done. But from a practical standpoint, there's still an equal amount of computation that has to happen to decode the frames and run the inverse DWTs, not to mention the additional load of debayering, color correction, and eventual transcoding. What the ZU+ does at 400fps, my laptop CPU struggles to undo at 0.25fps. So implementing decode and inverse DWT in a GPU-accelerated way just became high-priority. Luckily, I remember that I do at least have an existing <a href="https://scolton.blogspot.com/2014/12/fun-with-pixel-shaders.html">GPU debayer and color correction solution</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/2IHg6gl4_4c" width="640"></iframe></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I had to do a little work to modernize it, but since laptop hardware has also gotten way better since I wrote it, it can scrub through 4K raw just fine. Hopefully, I can implement the decode and IDWT there as well, to save the 4s per frame it currently takes to do those on the CPU. I'll also need the ZU+ to be able to do it, for preview and playback, but since it only has to run at 30fps that should be much easier than the forward direction.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Besides that, there are a few side-challenges I still have to take on:</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ol>
<li>The HDMI output, so I can see what the hell I'm doing. This will involve some amount of decoding and inverse DWT, at least of the 3rd and maybe 2nd wavelet stage, to generate a usable preview image with a menu and status overlay. I don't have the ability to output 4K, so a 1080p preview will have to suffice.<br /> </li>
<li>USB mass-storage device access to the SSD. As much as I love my <a href="https://www.amazon.com/Sabrent-EC-NVME-Aluminum-Enclosure-Nvme/dp/B07K4TZQ7D">Sabrent NVMe SSD enclosure</a>, I fear I am approaching the insertion/removal cycle limit on this drive. I have already demonstrated <a href="https://scolton.blogspot.com/2019/11/zynq-ultrascale-superspeed-ram-dumping.html">USB 3.0-speed mass storage device access to ZU+ RAM</a>, so this should be a simple SCSI bridge project. Bonus points if I can implement a few custom SCSI commands to start/stop recording or do other useful control tasks.<br /> </li>
<li>An alternative programmable logic configuration for the CMV12000's subsampled read-out mode. In this mode, the X and Y read-out skips every other 2x2 Bayer block, for a maximum resolution of 2048x1536 at a frame rate of 1050fps. Or an even more interesting 1472fps at 1080p. Because this mode reads in four rows in parallel, it will require more Stage 1 <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#h26">Horizontal DWT Cores</a>, which might be tough on the XCZU4.</li>
</ol>
<div>
But having the full capture pipeline in place is a good milestone, and I'm happy that it's pretty close to what I had in mind back in <a href="https://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">June</a>. Some of the details changed as I learned about the capabilities and limitations of the ZU+, but the high-level architecture is as-planned. Being able to capture 3.8Gpx/s continuously (for less than 5nJ/px, too) is something that I think is new and only recently possible with the latest generation SSDs and FPGA hardware working together. Keeping an eye on new components and trying to figure out interesting corner cases where they might be useful is something I enjoy, so this was a fun challenge.</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com3tag:blogger.com,1999:blog-8200098102909041178.post-34446911695260298692019-11-29T19:39:00.003-05:002021-10-08T17:49:35.867-04:00Zynq Ultrascale+ FatFs and Direct Speed Tests with Bare Metal NVMe via AXI-PCIe Bridge<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAsdsZd7roWIjycpWXHPF8M2p2Vnb86JDFHxi2-vYBcrG8Td4zn15hnbRMOJVYypUVOMX_dGF3TRiELwtkkjt94jDn_-yOVz9tlntT7Yepu0EGiX6fJEy9oa1xpmBjOIqyiLtqiqHZqMI/s1600/c99.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAsdsZd7roWIjycpWXHPF8M2p2Vnb86JDFHxi2-vYBcrG8Td4zn15hnbRMOJVYypUVOMX_dGF3TRiELwtkkjt94jDn_-yOVz9tlntT7Yepu0EGiX6fJEy9oa1xpmBjOIqyiLtqiqHZqMI/s640/c99.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Blue wire PCIe REFCLK still hanging in there...</td></tr>
</tbody></table>
It's time to return to the problem of <a href="https://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">sinking 1GB/s</a> of data onto an NVMe drive from a Zynq Ultrascale+ SoC. Last time, I <a href="https://scolton.blogspot.com/2019/07/benchmarking-nvme-through-zynq.html">benchmarked</a> the Xilinx Linux drivers and found that they were fast, but not quite fast enough. In the comments of that post, there were many good suggestions for how to make up the difference without having to resort to a hardware accelerator. The consensus is that the hardware, namely the stock AXI-PCIe bridge, should be fast enough.<br />
<br />
While a lot of the suggestions were ways to speed up the data transfer in Linux, and I have no doubt those would work, I also just don't want or need to run Linux in this application. The <a href="https://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">sensor input</a> and <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html">wavelet compression</a> modules are entirely built in Programmable Logic (PL), with only a minimal interface to the Processing System (PS) for configuration and control. So, I'm able to keep my entire application in the 256KB On-Chip Memory (OCM), leaving the external DDR4 RAM bandwidth free for data transfer.<br />
<br />
After compression, the data is already in the DDR4 RAM where it should be visible to whatever DMA mechanism is responsible for transferring data to an NVMe drive. As <a href="https://www.blogger.com/profile/16491915174390340818">Ambivalent Engineer</a> points out in the comments:<br />
<blockquote class="tr_bq">
<span style="color: #f9cb9c;">It should be possible to issue commands directly to the NVMe from software by creating a command and completion queue pair and writing directly to the command queue.</span></blockquote>
In other words, write a bare metal NVMe driver to interface with the AXI-PCIe bridge directly for initiating and controlling data transfers. This seems like a good fit, both to this specific application and to my general proclivity, for better or worse, to move to lower-level code when I get stuck. A good place to start is by exploring the functionality of the AXI-PCIe bridge itself.<br />
<h4>
AXI-PCIe Bridge</h4>
<div>
Part of the reason it took me a while to understand the AXI-PCIe bridge is that it has many names. The version for Zynq-7000 is called <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie.htm">AXI Memory Mapped to PCI Express (PCIe) Gen2</a>, and is covered in <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie.html#documentation">PG055</a>. The version for Zynq Ultrascale is called <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie_gen3.html">AXI PCI Express (PCIe) Gen 3 Subsystem</a>, and is covered in <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie_gen3.html#documentation">PG194</a>. And the version for Zynq Ultrascale+ is called <a href="https://www.xilinx.com/products/intellectual-property/pcie-dma.html">DMA for PCI Express (PCIe) Subsystem</a>, and is nominally covered in <a href="https://www.xilinx.com/products/intellectual-property/pcie-dma.html#documentation">PG195</a>. But, when operated in bridge mode, as it will be here, it's actually still documented in <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie_gen3.html#documentation">PG194</a>. I'll be focusing on this version. </div>
<div>
<br /></div>
<div>
Whatever the name, the block diagram looks like this:</div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtwnk5a4cLmOrKn8X9wgSp40DuhRA2CjpjpmL4k-d49tgS9-iD-Cgyg-u8hKgUZzuXazH6MvxIaUQyp2kFB9sGCH-8tPUGf8qta2m7GiN80jI3uoyT8h0bJysh6ZAbIojeoPvupf1ww_E/s1600/c91.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="940" data-original-width="1600" height="374" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtwnk5a4cLmOrKn8X9wgSp40DuhRA2CjpjpmL4k-d49tgS9-iD-Cgyg-u8hKgUZzuXazH6MvxIaUQyp2kFB9sGCH-8tPUGf8qta2m7GiN80jI3uoyT8h0bJysh6ZAbIojeoPvupf1ww_E/s640/c91.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">AXI-PCIe Bridge Root Port block diagram, adapted from <a href="https://www.xilinx.com/products/intellectual-property/axi_pcie_gen3.html#documentation">PG194</a> Figure 1.</td></tr>
</tbody></table>
</div>
<div style="text-align: left;">
The <span style="color: #9fc5e8;">AXI-Lite Slave Interface</span> is straightforward, allowing access to the bridge control and configuration registers. For example, the PHY Status/Control Register (offset <span style="color: #9fc5e8;">0x144</span>) has information on the PCIe link, such as speed and width, that can be useful for debugging. When the bridge is configured as a Root Port, as it must be to host an NVMe drive, this address space also provides access to the PCIe Configuration Space of both the Root Port itself, at offset <span style="color: #9fc5e8;">0x0</span>, and the enumerated Endpoint devices, at other offsets.</div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGf_rJTQbXh1O2mEVOqXc6wOj_LblHgR_a3K72jGi06atH91qKJ-ayQbbezbtzq_SE_Fodugxz9vyb9L3JEUHsdqH3OH_tP-PoJU-2RMSP0jg-KxAS7VijcUNSXxAAOAzJ3Jyj3pUHn_c/s1600/c92.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1087" data-original-width="1600" height="433" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGf_rJTQbXh1O2mEVOqXc6wOj_LblHgR_a3K72jGi06atH91qKJ-ayQbbezbtzq_SE_Fodugxz9vyb9L3JEUHsdqH3OH_tP-PoJU-2RMSP0jg-KxAS7VijcUNSXxAAOAzJ3Jyj3pUHn_c/s640/c92.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">PCIe Confinguration Space layout, adapted from <a href="https://www.xilinx.com/products/intellectual-property/pcie4-ultrascale-plus.html#documentation">PG213</a> Table 2-35.</td></tr>
</tbody></table>
If the NVMe drive has successfully enumerated, its Endpoint PCIe Configuration Space will be mapped to some offset in the <span style="color: #9fc5e8;">AXI-Lite Slave</span> register space. In my case, with no switch involved, it shows up as Bus 1, Device 0, Function 0 at offset <span style="color: #9fc5e8;">0x1000</span>. Here, it's possible to check the Device ID, Vendor ID, and Class Codes. Most importantly, the BAR0 register holds the PCIe memory address assigned to the device. <span style="color: yellow;">The AXI address assigned to BAR0 in the Address Editor in Vivado is mapped to this PCIe address by the bridge.</span></div>
<div>
<br /></div>
<div>
Reads from and writes to the AXI BAR0 address are done through the <span style="color: #ffe599;">AXI Slave Interface</span>. This is a full AXI interface supporting burst transactions and a wide data bus. In another class of PCIe device, it might be responsible for transferring large amounts of data to the device through the BAR0 address range. But for an NVMe drive, BAR0 just provides access to the NVMe Controller Registers, which are used to set up the drive and inform it of pending data transfers.</div>
<div>
<br /></div>
<div>
The <span style="color: #c27ba0;">AXI Master Interface</span> is where <i>all</i> NVMe data transfer occurs, for both reads and writes. One way to look at it is that the drive itself contains the DMA engine, which issues memory reads and writes to the system (AXI) memory space through the bridge. The host requests that the drive perform these data transfers by submitting them to a queue, which is also contained in system memory and accessed through this interface.</div>
<h4>
Bare Metal NVMe</h4>
<div>
Fortunately, NVMe is an open standard. The <a href="https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf">specification</a> is about 400 pages, but it's fairly easy to follow, especially with help from <a href="https://www.flashmemorysummit.com/English/Collaterals/Proceedings/2013/20130812_PreConfD_Marks.pdf">this tutorial</a>. The NVMe Controller, which is implemented on the drive itself, does most of the heavy lifting. The host only has to do some initialization and then maintain the queues and lists that control data transfers. It's worth looking at a high-level diagram of what should be happening before diving in to the details of how to do it:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMp4VLATTAXxAGJLZYnl9KcKh7y3AxcEx9IZsOXAsPt0dZVKiHQrjGjwxwfiPCeSQcURJ0JfHzfjTJIhUzBpw9Tqw9y2QhdNQ_mp5Dm95beMYVu3Q2A8J0iDFz-K2Heue4JWmr0nCnkhY/s1600/c93.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="984" data-original-width="1600" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMp4VLATTAXxAGJLZYnl9KcKh7y3AxcEx9IZsOXAsPt0dZVKiHQrjGjwxwfiPCeSQcURJ0JfHzfjTJIhUzBpw9Tqw9y2QhdNQ_mp5Dm95beMYVu3Q2A8J0iDFz-K2Heue4JWmr0nCnkhY/s400/c93.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">System-level look at NVMe data flow, with primary data streaming <i>from</i> a source <i>to</i> the drive.</td></tr>
</tbody></table>
<div style="text-align: left;">
After BAR0 is set, the host has access to the NVMe drive's Controller Registers through the bridge's <span style="color: #ffe599;">AXI Slave Interface</span>. They are just like any other device/peripheral control registers, used for low-level configuration, status, and control of the drive. The register map is defined in the <a href="https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf">NVMe Specification</a>, Section 2.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
One of the first things the host has to do is allocate some memory for the <span style="color: orange;">Admin Submission Queue</span> and <span style="color: orange;">Admin Completion Queue</span>. A Submission Queue (SQ) is a circular buffer of commands submitted to the drive by the host. It's written by the host and read by the drive (via the bridge <span style="color: #c27ba0;">AXI Master Interface</span>). A Completion Queue (CQ) is a circular buffer of notifications of completed commands from the drive. It's written by the drive (via the bridge <span style="color: #c27ba0;">AXI Master Interface</span>) and read by the host. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The <span style="color: orange;">Admin SQ/CQ</span> are used to submit and complete commands relating to drive identification, setup, and control. They can be located anywhere in system memory, as long as the bridge has access to them, but in the diagram above they're shown in the external DDR4. The host software notifies the drive of their address and size by setting the relevant Controller Registers (via the bridge <span style="color: #ffe599;">AXI Slave Interface</span>). After that, the host can start to submit and complete admin commands:</div>
<div style="text-align: left;">
</div>
<ol>
<li>The host software writes one or more commands to the <span style="color: orange;">Admin SQ</span>.</li>
<li>The host software notifies the drive of the new command(s) by updating the <span style="color: orange;">Admin SQ</span> doorbell in the Controller Registers through the bridge <span style="color: #ffe599;">AXI Slave Interface</span>.</li>
<li>The drive reads the command(s) from the Admin SQ through the bridge <span style="color: #c27ba0;">AXI Master Interface</span>.</li>
<li>The drive completes the command(s) and writes an entry to the <span style="color: orange;">Admin CQ</span> for each, through the bridge <span style="color: #c27ba0;">AXI Master Interface</span>. Optionally, an interrupt is triggered.</li>
<li>The host reads the completion(s) and updates the <span style="color: orange;">Admin CQ</span> doorbell in the Controller Registers, through the <span style="color: #ffe599;">AXI Slave Interface</span>, to tell the drive where to place the next completion.</li>
</ol>
<div>
In some cases, an admin command may request identification or capability data from the drive. If the data is too large to fit in the <span style="color: orange;">Admin CQ</span> entry, the command will also specify an address to which to write the requested data. For example, during initialization, the host software requests the Controller Identification and Namespace Identification structures, described in the <a href="https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf">NVMe Specification</a>, Section 5.15.2. These contain information about the capabilities, size, and low-level format (below the level of file systems or even partitions) of the drive. The space for these <span style="color: #3d85c6;">IDs</span> must also be allocated in system memory before they're requested.</div>
<div>
<br /></div>
<div>
Within the <span style="color: #3d85c6;">IDs</span> is information that indicates the Logical Block (LB) size, which is the minimum addressable memory unit in the non-volatile memory. 512B is typical, although some drives can also be formatted for 4KiB LBs. Many other variables are given in units of LBs, so it's important for the host to grab this value. There's also a maximum and minimum page size, defined in the Controller Registers themselves, which applies to system memory. It's up to the host software to configure the actual system memory page size in the Controller Registers, but it has to be between these two values. 4KiB is both the absolute minimum and the typical value. It's still possible to address system memory in smaller increments (down to 32-bit alignment); this value just affects how much can be read/written per page entry in an I/O command or PRP List (see below).</div>
<div>
<br /></div>
<div>
Once all identification and configuration tasks are complete, the host software can then set up one or more I/O queue pairs. In my case, I just want one <span style="color: #76a5af;">I/O SQ</span> and one <span style="color: #76a5af;">I/O CQ</span>. These are allocated in system memory, then the drive is notified of their address and size via admin commands. The <span style="color: #76a5af;">I/O CQ</span> must be created first, since the <span style="color: #76a5af;">I/O SQ</span> creation references it. Once created, the host can start to submit and complete I/O commands, using a similar process as for admin commands.</div>
<div>
<br /></div>
<div>
I/O commands perform general purpose writes (from system memory to non-volatile memory) or reads (from non-volatile memory to system memory) over the bridge's <span style="color: #c27ba0;">AXI Master Interface</span>. If the data to be transferred spans more than two memory pages (typically 4KiB each), then a Physical Region Page (PRP) List is created along with the command. For example, a write of 24 512B LBs starting in the middle of a 4KiB page might reference the data like this:</div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBKgxAtAOvgkyVwh-qT3i3fhtYWuXwTvo2B70-YxMKr7Qr0fqGmSVE1n8vOPK0zeTLnG-eDSgOzXfqNig_eEMRTBL2_qh80rY70PTzDdxowlTr2blgi9kiPklyGg0VOrW3Ny9BE4KDGoQ/s1600/c94.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="616" data-original-width="1600" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBKgxAtAOvgkyVwh-qT3i3fhtYWuXwTvo2B70-YxMKr7Qr0fqGmSVE1n8vOPK0zeTLnG-eDSgOzXfqNig_eEMRTBL2_qh80rY70PTzDdxowlTr2blgi9kiPklyGg0VOrW3Ny9BE4KDGoQ/s640/c94.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A PRP List is required for data transfers spanning more than two memory pages.</td></tr>
</tbody></table>
</div>
<div>
The first PRP Address in the I/O command can have any 32-bit-aligned offset within a page, but subsequent addresses must be page-aligned. The drive knows whether to expect a PRP Address or PRP List Pointer in the second PRP field of the I/O command based on the amount of data being transferred. It will also only pull as much data as is needed from the last page on the list to reach the final LB count. There is no requirement that the pages in the PRP list be contiguous, so it can also be used as a scatter-gather with 4KiB granularity. The PRP List for a particular command must be kept in memory until it's completed, so some kind of PRP Heap is necessary if multiple commands can be in flight.<br />
<br />
Some (most?) drives also have a Volatile Write Cache (VWC) that buffers write data. In this case, an I/O write completion may not indicate that the data has been written to non-volatile memory. An I/O flush command forces this data to be written to non-volatile memory before a completion entry is written to the <span style="color: #76a5af;">I/O CQ</span> for that flush command.</div>
<div>
<br /></div>
<div>
That's about it for things that are described explicitly in the specification. <span style="color: yellow;">Everything past this point is implementation detail that is much more application-specific.</span></div>
<div>
<br /></div>
<div>
A key question the host software NVMe driver needs to answer is whether or not to wait for a particular completion before issuing another command. For admin commands that run once during initialization and are often dependent on data from previous commands, it's fine to always wait. For I/O commands, though, it really depends. I'll be using write commands as an example, since that's my primary data direction, but there's a symmetric case for reads.</div>
<div>
<br /></div>
<div>
If the host software issues a write command referencing a range of data in system memory and then immediately changes the data, without waiting for the write command to be completed, then the write may be corrupted. To prevent this, the software could:</div>
<div>
<ol>
<li>Wait for completion before allowing the original data to be modified. (Maybe there are other tasks that can be done in parallel.)</li>
<li>Copy the data to an intermediate buffer and issue the write command referencing that buffer instead. The original data can then be modified without waiting for completion.</li>
</ol>
<div>
Both could have significant speed penalties. The copy option is pretty much out of the question for me. But usually I can satisfy the first constraint: If the data is from a stream that's being buffered in memory, the host software can issue NVMe write commands that consume one end of the stream while the data source is feeding in new data at the other end. With appropriate flow control, these write commands don't have to wait for completion.</div>
</div>
<div>
<br /></div>
<div>
My "solution" is just to push the decision up one layer: the driver <i>never</i> blocks on I/O commands, but it can inform the application of the I/O queue backlog as the slip between the queues, derived from sequentially-assigned command IDs. If a particular process thinks it can get away without waiting for completions, it can put more commands in flight (up to some slip threshold).<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHecbogWeVRhWoC_Adaxb2RlcqauE0kKnFEbhqxJnarSYeJXAh32g-kSAaaEWzqeMaH-av8FzBSTWU3CMNu5-kTaGqQJCUfbOOLqMK6bsHzibjVtvXNvaVPwMGjzG2mXyOcbfHTOHqMDU/s1600/c95.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="889" data-original-width="1600" height="354" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHecbogWeVRhWoC_Adaxb2RlcqauE0kKnFEbhqxJnarSYeJXAh32g-kSAaaEWzqeMaH-av8FzBSTWU3CMNu5-kTaGqQJCUfbOOLqMK6bsHzibjVtvXNvaVPwMGjzG2mXyOcbfHTOHqMDU/s640/c95.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">An example showing the driver ready to submit Command ID 72, with the latest completion being Command ID 67. The doorbells always point to the next free slot in the circular buffer, so the entry there has the oldest ID.</td></tr>
</tbody></table>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
<div>
I'm also totally fine with polling for completions, rather than waiting for interrupts. Having a general-purpose completion polling function that takes as an argument a maximum number of completions to process in one call seems like the way to go. <a href="https://github.com/nvmedirect/nvmedirect/blob/master/library/lib_nvmed.c#L246">NVMeDirect</a>, <a href="https://github.com/spdk/spdk/blob/9a25fc12bb7585c9cbcc8f642e38ecc879313a32/lib/nvme/nvme_pcie.c#L2066">SPDK</a>, and <a href="https://github.com/coreboot/depthcharge/blob/master/src/drivers/storage/nvme.c#L222">depthcharge</a> all take this approach. (All three are good open-source reference for light and fast NVMe drivers.)</div>
<div>
<br /></div>
<div>
With this set up, I am able to run a speed test by issuing read/write commands for blocks of data as fast as possible by trying to keep the I/O slip at a constant value:</div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC6CzDrS_LwqEz1KsgWYCvWRyaYmtgPvPH41MoQJN9cOH7eftSyVXCvG0oLp0XgTEVxINcSMeAVdf6GDKAzTxRJjQyf-jMyzeTix0c9x9WWL6Kl35pKxB_N8zY44FMCS7ENO4NvsyXxBM/s1600/c96.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="671" data-original-width="1600" height="268" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC6CzDrS_LwqEz1KsgWYCvWRyaYmtgPvPH41MoQJN9cOH7eftSyVXCvG0oLp0XgTEVxINcSMeAVdf6GDKAzTxRJjQyf-jMyzeTix0c9x9WWL6Kl35pKxB_N8zY44FMCS7ENO4NvsyXxBM/s640/c96.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Speed test for raw NVMe write/read on a 1TB Samsung 970 Evo Plus.</td></tr>
</tbody></table>
For smaller block transfers, the bottleneck is on my side, either in the driver itself or by hitting a limit on the throughput of bus transactions somewhere in the system. But for larger block transfers (32KiB and above) the read and write speeds split, suggesting that the drive becomes the bottleneck. And that's totally fine with me, since it's hitting <span style="color: yellow;">64% (write) and 80% (read) of the maximum theoretical PCIe Gen3 x4 bandwidth</span>.</div>
<div>
<br /></div>
<div>
Sustained write speeds begin to drop off after about 32GiB. The Samsung Evo SSDs have a feature called TurboWrite that uses some fraction of the non-volatile memory array as fast <a href="https://en.wikipedia.org/wiki/Multi-level_cell#Single-level_cell">Single-Level Cell</a> (SLC) memory to buffer writes. Unlike the VWC, this is still non-volatile memory, but it gets transferred to more compact <a href="https://en.wikipedia.org/wiki/Multi-level_cell">Multi-Level Cell</a> (MLC) memory later since it's slower to write multi-level cells. The 1TB drive that I'm using has around 42GB of TurboWrite capacity according to <a href="https://www.relaxedtech.com/reviews/samsung/970-evo-plus/">this review</a>, so a drop off in sustained write speeds after 32GiB makes sense. Even the sustained write speed is 1.7GB/s, though, which is more than fast enough for my application.<br />
<br />
A bigger issue with sustained writes might be getting rid of heat. This drive draws about 7W during max speed writing, which nearly doubles the total dissipated power of the whole system, probably making a fan necessary. Then again, at these write speeds a 0.2kg chunk of aluminum would only heat up about 25ºC before the drive is full... In any case, the drive will also need a good conduction path to the rear enclosure, which will act as the heat sink.</div>
<h4>
FatFs</h4>
</div>
<div>
I am more than content with just dumping data to the SSD directly as described above and leaving the task of organizing it to some later, non-time-critical process. But, if I can have it arranged neatly into files on the way in, all the better. I don't have much overhead to spare for the file system operations, though. Luckily, ChaN gifted the world <a href="http://elm-chan.org/fsw/ff/00index_e.html">FatFs</a>, an ultralight FAT file system module written in C. It's both tiny and fast, since it's designed to run on small microcontrollers. An ARM Cortex-A53 running at 1.2GHz is <i>certainly not</i> the target hardware for it. But, I think it's still a good fit for a very fast bare metal application.<br />
<br />
FatFs supports exFAT, but using exFAT still requires a license from Microsoft. I think I can instead operate right on the limits of what FAT32 is capable of:<br />
<ul>
<li>A maximum of 2^32 LBs. For 512B LBs, this supports up to a 2TiB drive. This is fine for now.</li>
<li>A maximum cluster size (unit of file memory allocation and read/write operations) of 128 LBs. For 512B LBs, this means 64KiB clusters. This is right at the point where I hit maximum (drive-limited) write speeds, so that's a good value to use.</li>
<li>A maximum file size of 4GiB. This is the limit of my RAM buffer size anyway. I can break up clips into as many files as I want. One file per frame would be convenient, but not efficient.</li>
</ul>
Linking FatFs to NVMe couldn't really get much simpler: FatFs's diskio.c device interface functions already request reads and writes in units of LBs, a.k.a. sectors. There's also a sync function that matches up nicely to the NVMe flush command. The only potential issue is that FatFs can ask for byte-aligned transfers, whereas NVMe only allows 32-bit alignment. My tentative understanding is that this can only happen via calls to <a href="http://elm-chan.org/fsw/ff/doc/read.html">f_read()</a> or <a href="http://elm-chan.org/fsw/ff/doc/write.html">f_write()</a>, so the application can guard against it.<br />
<br />
For file system operations, FatFs reads and writes single sectors to and from a working buffer in system memory. It assumes that the read or write is complete when the <a href="http://elm-chan.org/fsw/ff/doc/dread.html">disk_read()</a> or <a href="http://elm-chan.org/fsw/ff/doc/dwrite.html">disk_write()</a> function returns, so the diskio.c interface layer has to wait for completion for NVMe commands issued as part of file system operations. To enforce this, but still allow high-speed sequential file writing from a data stream, I check the address of the <a href="http://elm-chan.org/fsw/ff/doc/dwrite.html">disk_write()</a> system memory buffer. If it's in OCM, I wait for completion. If it's in DDR4, I allow slip. For now, I wait for completion on all <a href="http://elm-chan.org/fsw/ff/doc/dread.html">disk_read()</a> calls, although a similar mechanism could work for high-speed stream reading. And of course, <a href="http://elm-chan.org/fsw/ff/doc/dioctl.html">disk_ioctl()</a> calls for CTRL_SYNC issue an NVMe flush command and wait for completion.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimnXOwQDVmRL9FaM3X8rcyliJqcN9GleRACl91fhNLZujrMpzvXtcTI9IBlfu95ESbZvqShU71EJNUFgkguFyyFWJZrjcilnifBBvYzWb43uD05QELAdgppnRvgN-f-zgvqDwkeLYZDN0/s1600/c97.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1155" data-original-width="1600" height="287" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimnXOwQDVmRL9FaM3X8rcyliJqcN9GleRACl91fhNLZujrMpzvXtcTI9IBlfu95ESbZvqShU71EJNUFgkguFyyFWJZrjcilnifBBvYzWb43uD05QELAdgppnRvgN-f-zgvqDwkeLYZDN0/s400/c97.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Interface between FatFs and NVMe through diskio.c, allowing stream writes from DDR4.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
I also clear the queue prior to a read to avoid unnecessary read/write turnarounds in the middle of a streaming write. This logic obviously favors writes over reads. Eventually, I'd like to make a more symmetric and configurable diskio.c layer that allows fast stream reading and writing. It would be nice if the application could dynamically flag specific memory ranges as streamable for reads or writes. But for now this is good enough for some write speed testing:</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrSEBVOk7RBvydhIJ_5ueAQmLEk2Hasnzlav8dUjd7KusLg_FhJuiEh7Y4JxqCDRE0rl8ozCXoruFgpRJ3_RLL8o2FoX6cvWQ0UYZ-UUcFrG1Phn75BaZgjhAU-1p204CjDUH7d8EnppM/s1600/c98.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="668" data-original-width="1600" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrSEBVOk7RBvydhIJ_5ueAQmLEk2Hasnzlav8dUjd7KusLg_FhJuiEh7Y4JxqCDRE0rl8ozCXoruFgpRJ3_RLL8o2FoX6cvWQ0UYZ-UUcFrG1Phn75BaZgjhAU-1p204CjDUH7d8EnppM/s640/c98.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Speed test for FatFs NVMe write on a 1TB Samsung 970 Evo Plus.</td></tr>
</tbody></table>
<div style="text-align: left;">
There's a very clear penalty for creating and closing files, since the process involves file system operations, including reads and flushes, that will have to wait for NVMe completions. But for writing sequentially to large (1GiB) files, it's still exceeding my 1GB/s requirement, even for total transfer sizes beyond the TurboWrite limit. So I think I'll give it a try, with the knowledge that I can fall back to raw writing if I really need to.</div>
</div>
<h4>
Utilization Summary</h4>
<div>
The good news is that the NVMe driver (not including the Queues, PRP Heap, and IDs) and FatFs together take up only about 27KB of system memory, so they should easily run in OCM with the rest of the application. At some point, I'll need to move the .text section to flash, but for now I can even fit that in OCM. The <a href="https://github.com/coltonshane/SSD_Test">source</a> is here, but be aware it is entirely a test implementation not at all intended to be a drop-in driver for any other application.<br />
<br />
The bad news is that the XCZU4 is now pretty much completely full...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikdQBoG5MfI1DHaQvQCQUamIizk6ojBcYeV_HQg25UOMJVD8jKQ1SHJyQLvcTmDhXsKYFGcKS-4ceZXHx6PKdp5HjMsP0KFZ5hA8meoNk3PZtrpX0nSE_QMIjEtsnmkj2JLmQnDQ7zuyk/s1600/c90.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1107" data-original-width="1361" height="520" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikdQBoG5MfI1DHaQvQCQUamIizk6ojBcYeV_HQg25UOMJVD8jKQ1SHJyQLvcTmDhXsKYFGcKS-4ceZXHx6PKdp5HjMsP0KFZ5hA8meoNk3PZtrpX0nSE_QMIjEtsnmkj2JLmQnDQ7zuyk/s640/c90.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">We're gonna need a bigger chip.</td></tr>
</tbody></table>
<div style="text-align: left;">
The AXI-PCIe bridge takes up 12960 LUTs, 17871 FFs, and 34 BRAMs. That's not even including the additional AXI Interconnects. The only real hope I can see for shrinking it would be to cut the <span style="color: #ffe599;">AXI Slave Interface</span> down to 32-bit, since it's only needed to access Controller Registers at BAR0. But I don't really want to go digging around in the bridge HDL if I can avoid it. I'd rather spend time optimizing my own cores, but I think no matter what I'll need more room for additional features, like decoding/HDMI preview and the subsampled 2048x1536 mode that might need double the number of Stage 1 Wavelet horizontal cores. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
So, I think now is the right time to switch over to the <a href="https://shop.trenz-electronic.de/en/TE0808-04-6BE21-A-UltraSOM-MPSoC-Module-with-Zynq-UltraScale-XCZU6EG-1FFVC900E-4-GB-DDR4">XCZU6</a>, along with the <a href="http://scolton.blogspot.com/2019/11/zynq-ultrascale-superspeed-ram-dumping.html">v0.2 Carrier</a>. It's pricey, but it's a big step up in capability, with twice the DDR4 and more than double the logic. And it's closer to impedance-matched to the cost of the sensor...if that's a thing. With the XCZU6, I think I'll have plenty of room to grow the design. It's also just generally easier to meet timing constraints with more room to move logic around, so the compile times will hopefully be lower.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Hopefully the next update will be with the whole 3.8Gpx/s continuous image capture pipeline working together for the first time!</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com68tag:blogger.com,1999:blog-8200098102909041178.post-89769829238129408352019-11-17T16:08:00.001-05:002019-11-25T12:16:32.573-05:00Zynq Ultrascale+ SuperSpeed RAM Dumping + v0.2 CarrierI've gotten a lot of mileage out of my v0.1 (very first version) camera PCB. Partly that's because there's not much to it; it's mostly just power supplies, connectors, and differential pairs. But I'm still surprised I haven't broken it yet, and it's only had some minor design issues. I also made a front enclosure for it with an E-mount flange stolen from a macro extension tube (<a href="https://www.amazon.com/gp/product/B01N7UB0KU/ref=ppx_od_dt_b_asin_title_s00?ie=UTF8&psc=1">Amazon's cheapest</a>, of course) and slots for some 1/4-20 T-nuts for tripod mounting.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLjgvsb6E-7Udc4-xDdPIKwXFy1MJl0NrkCrzUM2ix-aztNmq6QEuUPDC5nOhBVaUnUdILHQVCsUGYXRDJvhwpyqHy_HaI_eNUaCMskv-vZrT_scpNfDgISrqJoMPXgin-31RRPjwqW9E/s1600/c51.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLjgvsb6E-7Udc4-xDdPIKwXFy1MJl0NrkCrzUM2ix-aztNmq6QEuUPDC5nOhBVaUnUdILHQVCsUGYXRDJvhwpyqHy_HaI_eNUaCMskv-vZrT_scpNfDgISrqJoMPXgin-31RRPjwqW9E/s640/c51.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Stealing an E-mount flange from a macro extension tube is maybe my favorite "Amazon's cheapest" hack so far. I'm not even sure how else to do it. Getting a custom CNC flange machined wouldn't be too bad, but what about the leaf springs?</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1SC4AKV3QSahAmJKE5ouPSH_V09YZxbkgT8rAWFAgnOl2mE8UoftJoMQUMwTH_nOVeyvNt_2dq8plbJV5Ld3V5nGQ1V5EwbkdwdflgxLbE7A2EsR12LzxILNm-fCWIdo4ZKV151wfyE0/s1600/c89.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1SC4AKV3QSahAmJKE5ouPSH_V09YZxbkgT8rAWFAgnOl2mE8UoftJoMQUMwTH_nOVeyvNt_2dq8plbJV5Ld3V5nGQ1V5EwbkdwdflgxLbE7A2EsR12LzxILNm-fCWIdo4ZKV151wfyE0/s640/c89.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">There are some sensor alignment features, but mostly the board just bolts to the back of the front enclosure. The 1/4-20 T-nuts allow for quick and dirty tripod mounting without having to worry about aluminum threads or Heli-Coils.</td></tr>
</tbody></table>
<div style="text-align: left;">
No real thought was given to connector placement, user interface, battery wiring/charging, cooling, or anything else other than having something to constrain the sensor and lens the right distance from each other and deal with the <a href="http://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">massive pixel throughput</a>. Still, it's been useful and reliable. At this point, I've tested most of the important hardware and am just about ready to make some functional improvements for v0.2.</div>
<br />
One important subsystem I hadn't tested yet, though, is the USB interface. It's not part of the capture pipeline, but it's important that it operate at USB 3.x speeds for reading image data off the SSD later. The Zynq Ultrascale+ has a built-in USB 3.0 PHY using PS-GTR transceivers at 5Gb/s. This isn't quite fast enough for 5:1 compressed image data at full frame rate, but it's more than fast enough for 30fps playback, or direct access for conversion and editing.<br />
<br />
At the moment, though, I'm mainly interested in USB 3.0 for reducing the amount of time it takes to get test image sequences out of the PS-side DDR4 RAM. I've so far been using XSCT commands to read blocks of RAM into a file (<span style="font-family: "courier new" , "courier" , monospace;">mrd -bin -file</span>) over JTAG, but this is limited by the 30MHz JTAG interface. That's a theoretical maximum, too. In practice, it takes several minutes to read out even a short image sequence, and up to <i>an hour</i> to dump the entire contents of the RAM (2GiB). This is all for mere <a href="https://www.youtube.com/watch?v=ep0a7_K0EIs">seconds of video</a>...<br />
<h4>
SuperSpeed RAM Dumping</h4>
To remedy this, I repurpose the <a href="https://github.com/Xilinx/embeddedsw/blob/master/XilinxProcessorIPLib/drivers/usbpsu/examples/xusb_poll_example.c">standalone ZU+ USB mass storage class example</a> to map most of the RAM as a virtual disk, then use a raw disk image reader (<a href="https://sourceforge.net/projects/win32diskimager/">Win32 Disk Imager</a>) to read it. This is pretty much what the example does anyway, so my modifications were very minor. So far, I've been able to run my application in On-Chip Memory (OCM), leaving the external DDR4 free for image capture. So, I have to explicitly place the virtual disk in DDR4 in the linker script:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLpvWOc8A5YXCBiSHY4AUfKCC4XE7-mvi_I_bQgmHKjJSmQB9tq1T5evhFkeJeap9Z5G2r94FzLnPdPDH_WJNowgPEhjjQw5NjFulXwIMvDxPHRXmZ9YI5uoNdxlq4OR_TNPyTtxqkNk/s1600/c82.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1248" data-original-width="1062" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeLpvWOc8A5YXCBiSHY4AUfKCC4XE7-mvi_I_bQgmHKjJSmQB9tq1T5evhFkeJeap9Z5G2r94FzLnPdPDH_WJNowgPEhjjQw5NjFulXwIMvDxPHRXmZ9YI5uoNdxlq4OR_TNPyTtxqkNk/s640/c82.png" width="544" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
In the application, the virtual disk array also needs to be correctly sized and assigned to the dedicated memory section using an __attribute__:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihF80LW-aUuVGjBekF-WuKlc2g1tTShwrxV9jmhq_uPV4BObG2Cg36qm7ZDSGqpkJpttxb9O1PZw8Qi9ITazyyBiH98nDu-D8SvYgAdpY-QQnCTk95veOEXas868HsTjJIfjwsMlQA2PI/s1600/c83.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="403" data-original-width="1303" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihF80LW-aUuVGjBekF-WuKlc2g1tTShwrxV9jmhq_uPV4BObG2Cg36qm7ZDSGqpkJpttxb9O1PZw8Qi9ITazyyBiH98nDu-D8SvYgAdpY-QQnCTk95veOEXas868HsTjJIfjwsMlQA2PI/s640/c83.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
With that small modification, the application (including the mass storage device driver) runs in OCM RAM, but references a virtual disk array based in external DDR4 at 0x20000000, which is where the image capture data starts. As with the original example, when plugged in to a host, the device shows up as a blank drive of the defined size. Windows asks to format it, but for now I just click Cancel and use Win32 Disk Imager to read the entire 1.25GiB. This copies the raw contents of the "disk" into a binary file, a process I'm all too familiar with from having to recover files from SD cards with corrupted file systems.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
But at first I wasn't getting a SuperSpeed (5Gb/s) connection; it was falling back to High-Speed (480Mb/s) through the external <a href="https://www.microchip.com/wwwproducts/en/USB3320">USB3320 PHY</a>. (An external USB 2.0 PHY is required on the ZU+, even for SuperSpeed operation.) To further troubleshoot, I took a look at the <a href="https://www.xilinx.com/html_docs/registers/ug1087/ug1087-zynq-ultrascale-registers.html#usb3_xhci___dcfg.html">DCFG</a> and <a href="https://www.xilinx.com/html_docs/registers/ug1087/ug1087-zynq-ultrascale-registers.html#usb3_xhci___dsts.html">DSTS</a> registers in the USB module. DCFG indicated a Device Speed of 3'b100 (SuperSpeed), but DSTS indicated a Connection Speed of 3'b000 (High-Speed). I figured this meant the PS-GTR link to the host was failing, and after some more poking around I found that its reference clock source was set to incorrect pins <i>and</i> frequency. In my case, I'm feeding it with a 100MHz reference clock on input 2, so I changed it accordingly:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrBaaiJnhvflDN5NgvLu9IiGSA5k9l3pYYVyyRIBVMiwiKSrMS9q0xLqHN6k2re5rghw4LlABnCzut59OKt3XzSOO8ANRQHG2o3j85iEJ0ezWvnz9-4NTgD_AKO3wEWa0T691dPKCPRe0/s1600/c84.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="955" data-original-width="1330" height="457" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrBaaiJnhvflDN5NgvLu9IiGSA5k9l3pYYVyyRIBVMiwiKSrMS9q0xLqHN6k2re5rghw4LlABnCzut59OKt3XzSOO8ANRQHG2o3j85iEJ0ezWvnz9-4NTgD_AKO3wEWa0T691dPKCPRe0/s640/c84.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
After that, I was able to get a SuperSpeed connection. As a formatted disk drive, I get sequential read speeds of around <span style="color: yellow;">300MB/s</span>. Through Win32 Disk Imager, I can read the entire 1.25GiB virtual disk in about seven seconds. So much better! To celebrate, I set off some steel wool fireworks with Bill Kerman. (Steel wool, especially the ultrafine variety, <a href="https://www.youtube.com/watch?v=oiWZYdr9Zvo">burns quite spectacularly</a>.)</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/TEHlCqnowWk" width="640"></iframe></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Since I've been putting off the task of NVMe writing, these are still just image sequences that can fit in the RAM buffer. In this case they're actually compressed about 11:1, well beyond my SSD writing requirement, mostly due to the relatively dark and low-contrast scene. The same <a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#quantizer">quantizer</a> settings in a brighter scene with more detail would yield a lower compression ratio. I did separate out the quantizer values for each subband, so I can experiment more with the quality/data rate trade-off.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The most noticeable defects aren't from wavelet compression, they're just the regular sensor defects. There's definitely some "black sun" artifact in the brightest sparks. There's also a rolling row offset that makes the dark background appear to flicker. I did switch to a different power supply for this test, which could be contributing more electrical noise. In any case, I definitely need to implement row noise correction. The combination of all-intraframe compression and a global shutter does make it pretty good for observing the sometimes crazy behavior of individual sparks, though:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD_y_EexyI2XnEFhFTpEF9zC_om9OQiNGuSjq22VruoLEpzEDbxaunMACw7our0HPmQZ16j-euzKNmYcwhMIaMvWIvPlGUt9pfRfMXOQ70gq9z1_Hb_p_fJRwOW6B7tAbtBQ4DttlTSQ0/s1600/c87.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhD_y_EexyI2XnEFhFTpEF9zC_om9OQiNGuSjq22VruoLEpzEDbxaunMACw7our0HPmQZ16j-euzKNmYcwhMIaMvWIvPlGUt9pfRfMXOQ70gq9z1_Hb_p_fJRwOW6B7tAbtBQ4DttlTSQ0/s640/c87.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This one was gently falling and then just decided to explode into a dozen pieces, shooting off at 20-30mph.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHqagvAxJU7Ez8zAnnlKYium8ksT1NSK9gczNgyATynMKtojRmGUUhdzP6aAYLCbxpZu-xPO2UdcLFY_SrXTwzeTwnnlmMRmSFQMV_md8-afslc8f3SDZbRky8ZhswfJplEi5okorX9qo/s1600/c88.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHqagvAxJU7Ez8zAnnlKYium8ksT1NSK9gczNgyATynMKtojRmGUUhdzP6aAYLCbxpZu-xPO2UdcLFY_SrXTwzeTwnnlmMRmSFQMV_md8-afslc8f3SDZbRky8ZhswfJplEi5okorX9qo/s640/c88.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My favorite, though, is this spark that gets flung off like a pair of binary stars. After a while, they decide to part ways and one goes flying up behind Bill's helmet. The comet-like tails are a motion artifact of the multi-slope exposure mode.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
Another thing I learned from this is that I probably need an <a href="https://en.wikipedia.org/wiki/Infrared_cut-off_filter">IR-cut filter</a>. I neglected to record some normal footage of the steel wool burning, but it's nowhere near as bright as it looks here. Much of that is just how human visual perception works. I tried to mitigate it somewhat by using the CMV12000's multi-slope exposure mode to rein in the highlights. But I think there's also some near-infrared adding to the brightness here. I'll have to get an external IR-cut filter to test this theory.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Although the image sequence transfer is 100x faster now, it still takes time to adjust settings and trigger the capture over JTAG. I would very much like to do everything over USB in the near future, at least until I have some form of UI on the camera itself. But I also don't really want to write a custom driver. I might try to abuse the mass storage device driver, since it's already working, by adding in some custom SCSI codes for control. This is also the device class I intend to use eventually as the host interface to the SSD, so I should get to know it well.</div>
<h4 style="clear: both; text-align: left;">
v0.2 Carrier</h4>
<div class="separator" style="clear: both; text-align: left;">
Controlling the camera over USB is not the most user-friendly way of doing things, as I know from wrangling drivers and APIs for <a href="http://scolton.blogspot.com/p/video.html#gs3">previous camera projects</a>. I could <i>maybe</i> see an exception where a Pixel 2 (modern Pixels don't have USB 3.0 anymore, because smartphone progress makes no fucking sense) hosts the camera, presenting a nice preview image and dedicated touch interface. But that's a large chunk of Android development that I don't want or know how to do.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Instead, I think it makes sense to stick to something extremely simple for now: an HDMI output and some buttons. I would love to have a touchscreen LCD, but they're huge time, money, power, and reliability sinks. They're also never bright enough, or if they are they kill the power and thermal budget. Better to just move the problem off-board, where it can be solved more flexibly depending on the scenario. At least that's what I'll tell myself.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
It seems like there are two main ways to do HDMI out from a Zynq SoC. The more modern Zynq Ultrascale+ development boards, like the <a href="https://www.xilinx.com/products/boards-and-kits/zcu106.html">ZCU106</a>, use a PL-side GTH transceiver to directly drive a TMDS retimer. This supports HDMI 2.0 (4K), but would rule out the cheaper <a href="https://shop.trenz-electronic.de/en/TE0803-02-04CG-1EA-MPSoC-Module-with-Xilinx-Zynq-UltraScale-ZU4CG-1E-2-GByte-DDR4-5.2-x-7.6-cm">TE0803 XCZU4</a> board, since its four PL-side transceivers are already in use for the SSD. The second method uses a dedicated HDMI transmitter like the <a href="https://scolton.blogspot.com/2019/07/benchmarking-nvme-through-zynq.html">Analog Devices ADV7513</a> as an external PHY, which interfaces to the Zynq over a wide logic-level pixel bus. Even though it only goes up to 1080p, this sounds more like what I want for now. I just need a reasonable preview image.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_zNjBlWocMWRy99HitBnHIK7VyMAQmPwcJnCGP7HNt4b3_45ikZDkKXYYiPKccf4t1MUGfDWQCetxn_o_hwmg8rzzkMLbVDsGUtXzUxEt7jlV5KyZSFLIINeZjuR3WT6MZGT-ESXTMAc/s1600/c85.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1004" data-original-width="1600" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_zNjBlWocMWRy99HitBnHIK7VyMAQmPwcJnCGP7HNt4b3_45ikZDkKXYYiPKccf4t1MUGfDWQCetxn_o_hwmg8rzzkMLbVDsGUtXzUxEt7jlV5KyZSFLIINeZjuR3WT6MZGT-ESXTMAc/s640/c85.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">HDMI output subsystem based on the ADV7513.</td></tr>
</tbody></table>
I had left a bunch of unused pins in the top right corner expecting to need a wide logic-level pixel bus, either for an LCD or an HDMI transmitter. The tricky part was finding room for the connector and IC. I decided to ditch the microSD card holder, which had a bad footprint anyway, to make the space. Without growing the board, I can fit a full-size (Type A) HDMI connector on the top side and the ADV7513 plus supporting components on the bottom. The TMDS lines do have to change layers once, but they're short and length-matched so I think they'll be okay.<br />
<div>
<br /></div>
<div>
At the same time, I also rerouted a good portion of the right edge of the board. The port I've been using for UART terminal access is gone, replaced by a more general-purpose optically-isolated I/O connector. This can still be used for terminal access, or as a trigger/sync signal. I also added a barrel jack connector for power/charge input. Finally, a 0.1" header off the back of the board has the battery power input and some unprotected I/O for two buttons, a rotary encoder, and a red "recording" LED on a separate board. This UI board would be mounted to the top face, right-hand side, where such things would typically be on a camera.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK_nyiva4J_ue4vRwUbO3A9Fo25_xjYJPTaeyPDN5yxt0DJaCBbpmk-unGJXt8w-Sxpacsjiwr2nf2dPzGf4-y6pKJ-ldvmyPtYDcZ3V5dFSdNi-SxlIkCJi8Gpt_5IBpvlCOFhAp6s9Y/s1600/c86.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1331" data-original-width="1600" height="532" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK_nyiva4J_ue4vRwUbO3A9Fo25_xjYJPTaeyPDN5yxt0DJaCBbpmk-unGJXt8w-Sxpacsjiwr2nf2dPzGf4-y6pKJ-ldvmyPtYDcZ3V5dFSdNi-SxlIkCJi8Gpt_5IBpvlCOFhAp6s9Y/s640/c86.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New right-edge connector layout and top face UI board header.</td></tr>
</tbody></table>
<div style="text-align: left;">
I consider this to be the bare minimum design for standalone functionality. It will need a simple menu and status overlay on the HDMI output. I'm also skipping any BMS or charge circuitry for now, so the battery must be self-contained (like this <a href="https://www.newegg.com/p/14R-008K-00136?item=9SIA2EY0ZU2243&source=region&nm_mc=knc-googlemkp-pc&cm_mmc=knc-googlemkp-pc-_-pla-tenergy-_-tools+-+batteries-_-9SIA2EY0ZU2243&gclid=CjwKCAiA_MPuBRB5EiwAHTTvMXIP10Q7hGOHyn7coa3zSFbn8DyuB8-qLV5kVlj3wlB9K4bK8iJy8xoCw2YQAvD_BwE&gclsrc=aw.ds">3-cell pack</a>) and charged by a CC/CV adapter. It's well-within the power range of USB C charging, so that could be an option in the future, but I don't think it's important enough for this revision.<br />
<br />
One of the reasons I don't mind doing more small iterations rather than trying to cram features into one big revision is that I have been able to get these boards relatively fast and cheap from <a href="https://jlcpcb.com/">JLCPCB</a>. Originally, I chose their service because they were the first and only place I found with a standard <a href="https://jlcpcb.com/client/index.html#/impedance">impedance-controlled stack-up</a>, complete with an online <a href="https://jlcpcb.com/client/index.html#/impedanceCalculation">calculator</a>. But it's also just the most economical way to get a six-layer impedance controlled board in under two weeks. Each one is around $30. Even including all the power supplies and interfaces, the board is really a minor cost compared to the sensor and SoM it carries.<br />
<br />
Other than that, there was only one minor fix that needed to be made regarding the SSD's PCIe reference clock. I had mistakenly assumed this could be generated or forwarded by the ZU+ out of one of its GT clock pairs. But this doesn't seem to be standard practice. Instead, the external clock generator distributes matching reference clocks to both the ZU+ GT clock input and the SSD. I hacked this on to v0.1 with some twisted pair blue wire surgery, but it was easy to reroute for v0.2. Aside from this, I didn't touch any of the differential pairs, or really any other part of the board. Well, I did add one more small component...but that'll be for much later.<br />
<br />
These boards should arrive in time for a Thanksgiving weekend soldering session. I plan to build up two this time: one monochrome and, if all goes well, finally, one <b><span style="color: red;">c</span><span style="color: lime;">o</span><span style="color: #0b5394;">l</span><span style="color: lime;">o</span><span style="color: red;">r</span></b>. Before then, I'd like to have at least some plan for the NVMe write...</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-77911048710796742462019-11-11T17:23:00.001-05:002019-11-12T16:58:46.006-05:00TinyCross: 4WD and Servoless RC ModeI finished building up the second <a href="http://scolton.blogspot.com/2018/09/tinycross-electron-control-unit.html">dual motor drive</a> for TinyCross, which means that the electronics and wiring have finally caught up to the mechanical build and both are 100% complete!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOx5cWvu-JENhLKtK94kXJV5Mb_LE90NNd5ItoWsaRSM8vC0VdiiUaIxPJ4skINc0Tak4zlhUTu7ZkJtA-1enRjIjX3cF_neOwCoCgzAtvQMQSM-G1JsYkL50L4ml-PZKp0DeLKfdtddw/s1600/tc76.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOx5cWvu-JENhLKtK94kXJV5Mb_LE90NNd5ItoWsaRSM8vC0VdiiUaIxPJ4skINc0Tak4zlhUTu7ZkJtA-1enRjIjX3cF_neOwCoCgzAtvQMQSM-G1JsYkL50L4ml-PZKp0DeLKfdtddw/s640/tc76.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
That's not to say that the project is 100% complete; there's still some testing to be done to bring it all the way up to full power, as well as some weight reduction and weatherproofing tasks. But there are no more parts sitting in bins waiting for installation. It should be at peak mass, too, which is good because it's 86lb (39kg) <i>without </i>batteries. The original target was 75lb (34kg) without batteries, but I will settle for 80lbs (36kg) if I can get there.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The second <a href="https://scolton.blogspot.com/2018/09/tinycross-electron-control-unit.html">TxDrive</a> went together with no issues, and the software is identical to the front wheel drive. I have both set at 80A right now, which gives a total force at the ground of 112lbf (51kgf). That's about the same peak force as the rebuilt <a href="https://scolton.blogspot.com/2013/10/tinykart-black-alien-power.html">"black" version of tinyKart</a>, which was <a href="https://www.youtube.com/watch?v=OuCMtIB5P0E">maybe too much</a> for that frame. But TinyCross is about 20% heavier (with the driver weight included) and 4WD, so it should be able to handle some more. I haven't seen any thermal issues at 4x80A - if anything, the motors run cooler now that all four are sharing the acceleration. Over the next few test drives, I'll work my way up toward the 120A target.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
But before that, there's something I've been wanting to try. I have an abundance of actuators and not that many degrees of freedom. I decided to borrow an idea from <a href="https://scolton.blogspot.com/2016/06/twitch-x-servoless-linkage-drive.html">Twitch X</a> to cash in some of this actuator surplus for one more degree of control, specifically automatic servoless steering. So, a free 1/3-scale RC car mode without adding any parts.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBcY_WJ0Upn1QfSDIJlmtMOZH-wlgnChqeLSuOH_ulqzyDSxjdsdUsRXGCZhygKYlp5W-UYCMtarUqYVJlJaFV6yLXlPJEQZVXd7y5PJmlaLSNTWcoscNhCTw_PDhogyf5G9nXoH_OYOM/s1600/tc77.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBcY_WJ0Upn1QfSDIJlmtMOZH-wlgnChqeLSuOH_ulqzyDSxjdsdUsRXGCZhygKYlp5W-UYCMtarUqYVJlJaFV6yLXlPJEQZVXd7y5PJmlaLSNTWcoscNhCTw_PDhogyf5G9nXoH_OYOM/s640/tc77.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Well okay, I do have to add the receiver.</td></tr>
</tbody></table>
<div style="text-align: left;">
The steering wheel board reads the throttle and steering PWM signals from a normal RC receiver. The throttle PWM gets directly mapped to a torque command for all four motors. The steering PWM sets a target angle for the steering wheel. The measured angle comes from an IMU, the secret part on the steering wheel board. (Yes, there are all sorts of issues with that...I honestly just don't want to run any more wires.) The angle error drives a feedback controller that outputs differential torque commands to the front motors. Not much to it, really.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk4gYRTy0w8qeI1wbRTv2GmJQ7hIC39pheEX5cekVYXflrXIJVeyiNaTjXFz7LGP5Ixyf8GiEcl0S-mwBmZZ74Fd0HK4R-TXAxvoHBm8iXlv6WNAcOIulHqJ-35zyc67InEow-u-M3QxY/s1600/tc79.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1390" data-original-width="1203" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk4gYRTy0w8qeI1wbRTv2GmJQ7hIC39pheEX5cekVYXflrXIJVeyiNaTjXFz7LGP5Ixyf8GiEcl0S-mwBmZZ74Fd0HK4R-TXAxvoHBm8iXlv6WNAcOIulHqJ-35zyc67InEow-u-M3QxY/s400/tc79.png" width="345" /></a></div>
I've also seen <i>so many</i> runaway robots and go-karts in my life that I consider it a must to have working failsafes for both radio loss of signal and receiver (PWM) disconnect. It's extra work but trust me, it's worth it! Anyway, time for a test drive:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/LCgn0YJ2SII" width="640"></iframe></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I wasn't sure how tightly I could tune the steering control loop, since there's a long chain of mechanical mush between the torque output at the motors and the sensor input at the steering wheel. But it works just fine. After a minute I forgot it wasn't really an RC car and tried some curb jumping. Just like Twitch X, the wheels do need traction to be able to control the steering angle. But then again, that is a necessary condition for steering anyway.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I don't actually think there's much point in a go-kart-sized RC car. But it's a short jump from that to an autonomous platform. It might also be useful to adjust the "feel" of the steering during normal driving. Mostly, I just like to abide by the Twitch X philosophy of using your existing actuators to do as much as possible.</div>
<div style="text-align: left;">
<br /></div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com7tag:blogger.com,1999:blog-8200098102909041178.post-37359832214845690122019-10-27T01:36:00.000-04:002019-11-15T14:31:03.240-05:00Real-Time Wavelet Compression for High Speed Video<div class="separator" style="clear: both; text-align: left;">
The next stop on the <a href="http://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">Freight Train of Pixels</a> is the wavelet compression engine. Previously, I built up the <a href="http://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">CMV12000 input module</a>, which turned out to be easier than I thought. The output of that module is a set of 64 10-bit pixels and one 10-bit control signal that update on a 60MHz pixel clock (px_clk). This is too much data to write directly to an NVMe SSD, so I want to compress it by about 5:1 in real-time on the <a href="https://shop.trenz-electronic.de/en/TE0803-02-04CG-1EA-MPSoC-Module-with-Xilinx-Zynq-UltraScale-ZU4CG-1E-2-GByte-DDR4-5.2-x-7.6-cm?c=452">XCZU4</a> Zynq Ultrascale+ SoC. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Wavelet compression seems like the right tool for the job, prioritizing speed and quality over compression ratio. It needs to run on the SoC's programmable logic (PL), where there's enough parallel computation and memory bandwidth for the task. This post will build up the theory and implementation of this wavelet compression engine, starting from the basics of the discrete wavelet transform and ending with encoded data streams being written to RAM. It's a bit of a long one, so I broke it into several sections:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#dwt">The Discrete Wavelet Transform</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#lifting">The Lifting Scheme</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#h26">Horizontal 2/6 DWT Core</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#v26">Vertical 2/6 DWT Core</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#extensions">Input Extensions?</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#quantizer">Quantizer</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#encoder">Variable-Length Encoder</a>]</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#wrap">Wrapping Up</a>] - First test video here.</div>
<div class="separator" style="clear: both; text-align: left;">
[<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html#info">More Information</a>] - Source here, plus some references.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<h4 style="clear: both; text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="dwt"></a>The Discrete Wavelet Transform</h4>
<div class="separator" style="clear: both; text-align: left;">
Suppose we want to store the row vector [150, 150, 150, 150, 150, 150, 150, 150]. All the values are less than 256, so eight bytes works. If this vector is truly from a random stream of bytes, that might be the best we can do. But real-world signals of interest are not random and a pattern of similar numbers reflects something of physical significance, like a stripe in a bar code. This fact can be leveraged to represent a structured signal more compactly. There are many ways to do this, but let's look specifically at the discrete wavelet transform (DWT).</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
A simple DWT could operate on adjacent pairs of data points, taking their sum (or average) and difference. This two-input/two-output operation would scan, without overlap, through the row vector to produce four averages and four differences, as shown below. Assuming no rounding has taken place, the resulting eight values <i>fully represent</i> the original data, since the process could be reversed. Furthermore, the process can be repeated in a binary fashion on just the averages:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwZpgLHcY7cj4HCx3hGCCTVf4eyPiMwhj-cCVDFYDz2FiIrxYzASqKbh7mluMQvawvkvHKUb680b5AobL_MrKerKz8PXwfKdP_3neuPwuXywuTEgfmi57uHpp2c4Vj5kmV24imZ8fnuGI/s1600/wv04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1498" data-original-width="860" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwZpgLHcY7cj4HCx3hGCCTVf4eyPiMwhj-cCVDFYDz2FiIrxYzASqKbh7mluMQvawvkvHKUb680b5AobL_MrKerKz8PXwfKdP_3neuPwuXywuTEgfmi57uHpp2c4Vj5kmV24imZ8fnuGI/s640/wv04.png" width="364" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
After three levels, all that remains is a single average and seven zeros, representing the lack of difference between adjacent data points at each stage above. This is an extreme example, but in general the DWT will concentrate the information content of a structured signal into fewer elements, paving the way for compression algorithms to follow.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
For image compression, it's possible to perform a 2D DWT by first transforming the data horizontally, then transforming the intermediate result vertically. This results in four outputs representing the average image as well as the high-frequency horizontal, vertical, and diagonal information. The entire process can then be repeated on the new 1/2-scale average image.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgATzF5QiQ3F45ftGOwZwsMa6gBSGla78IKHX-ER9S67YvHkEUve8dxa3twvEFMQqXtSCSn6hKBS48_YeuAYK2doombOu9wgQprp_hGPW3a3xnPgiEWe3NZyJ9Qx-VecRTYY36BdMoqRIE/s1600/wv15.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="733" data-original-width="1297" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgATzF5QiQ3F45ftGOwZwsMa6gBSGla78IKHX-ER9S67YvHkEUve8dxa3twvEFMQqXtSCSn6hKBS48_YeuAYK2doombOu9wgQprp_hGPW3a3xnPgiEWe3NZyJ9Qx-VecRTYY36BdMoqRIE/s640/wv15.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A three-stage 2D Haar DWT output. Green indicates zero difference outputs.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
The DWT discussed above uses the simplest possible wavelet, the <a href="https://en.wikipedia.org/wiki/Haar_wavelet">Haar wavelet</a>, which only looks at two adjacent values for both the sum/average and the difference calculation. While this is extremely fast, it has relatively low curve-fitting ability. Consider what happens if the Haar DWT is performed on a ramp signal:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRPda4t2EDw8OuijyuagDnU9xnGEGIspbPHa8O_Za4kvIPjSsvil2Qx8ktVi0G5JLSth7kFSgUNNO9j1FJopjwMSCS4f-vu_-aO1WfFzybI8N0YwneimQ-Io1wGJv1WvoWyPqxUYHFbMg/s1600/wv05.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1022" data-original-width="1219" height="335" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRPda4t2EDw8OuijyuagDnU9xnGEGIspbPHa8O_Za4kvIPjSsvil2Qx8ktVi0G5JLSth7kFSgUNNO9j1FJopjwMSCS4f-vu_-aO1WfFzybI8N0YwneimQ-Io1wGJv1WvoWyPqxUYHFbMg/s400/wv05.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
Instead of zeros, the ramp input produces a constant difference output. It's still smaller than the original signal, but not as good for compression as all zeros. It's possible to use more complex wavelets to capture higher-order signal structure. For example, a more complex wavelet may compute the deviation from the local slope instead of the immediate difference, which is back to zero for a ramp input:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPakDkVpgF3YAzOmPDPgnarm4xPM-QvFunPdDXUGVbR0cEm6qADJ4_LaLlW3nHKUww0q4-AOOVVtnhzskReuPTqL12ioVXO2CyS0fy9ZpbuuwSUcwEtTnY7H1FQEDZJBcWfBGdovVO2NA/s1600/wv06.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="996" data-original-width="1216" height="327" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPakDkVpgF3YAzOmPDPgnarm4xPM-QvFunPdDXUGVbR0cEm6qADJ4_LaLlW3nHKUww0q4-AOOVVtnhzskReuPTqL12ioVXO2CyS0fy9ZpbuuwSUcwEtTnY7H1FQEDZJBcWfBGdovVO2NA/s400/wv06.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
To compute the "local slope", some number of data points before and after the pair being processed are needed. The sum/average operation may also be more complex and use more than two data points. One classification system for wavelets is based on how many points they use for the sum and difference operations, their <a href="https://en.wikipedia.org/wiki/Support_(mathematics)">support</a>. The Haar wavelet would be classified as 2/2 (for sum/difference), and is unique in that it doesn't require any data beyond the immediate pair being processed. A wavelet with larger support can usually fit higher-order signals with fewer non-zero coefficients.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The better signal-fitting ability of wavelets with larger support comes at a cost, since each individual operation requires more data and more computation. Now is a good time to introduce the 2/6 wavelet that will be the focus of most of this post. It uses two data points for the sum/average, just like the Haar wavelet, but six for the difference. One way to look at it is as a pair of weighted sums being slid across the data:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFIp4G1M8hAn8HgfzBXQfPt47-gRkupwMS7FO8WPEmapqk5Vu5v_GkacSlNUokARGdYH6kaVqZSqjgXV62Ui4HpfD5_gCxwhrgcKVPYKf4NoM18emrj_Qoqm_29Xp4Pts1Zy3vFa8IewE/s1600/wv08.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="796" data-original-width="917" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFIp4G1M8hAn8HgfzBXQfPt47-gRkupwMS7FO8WPEmapqk5Vu5v_GkacSlNUokARGdYH6kaVqZSqjgXV62Ui4HpfD5_gCxwhrgcKVPYKf4NoM18emrj_Qoqm_29Xp4Pts1Zy3vFa8IewE/s320/wv08.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
The two-point sum is a straightforward moving average. The six-point difference is a little more involved: the four outer points are used to establish the local slope, which is then subtracted from the immediate difference calculated from the two inner points. This results in zero difference for constant, ramp, and even second-order signals. Although it's more work than the Haar wavelet, the computational requirement is still relatively low thanks to weights that are all powers of two.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The 2/6 wavelet is used by the <a href="https://gopro.github.io/cineform-sdk/">CineForm</a> codec, which was open-sourced by GoPro in 2017. The <a href="https://gopro.github.io/cineform-sdk/">GitHub page</a> has a great explanation of how wavelet compression works, along with the full SDK and example code. There's also a very easy to follow <a href="https://github.com/gopro/cineform-sdk/tree/master/Example/WaveletDemo">stripped-down C example</a> that compresses a single grayscale image. If you want to explore the entire history of CineForm, which overlaps in many ways with the history of wavelet-based video compression in general, it's definitely worth reading through the <a href="https://cineform.blogspot.com/">GoPro/CineForm Insider</a> blog.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Besides the added computation, the wavelets other than the Haar also need data from outside the immediate input pair. The 2/6 wavelet, for example, needs two more data points to the left and to the right of the input pair. This creates a problem at the first and last input pair of the vector, where two of the data points needed for the difference calculation aren't available. Typically, the vector is extended by padding it with data that is in some way an extension of the nearby signal. This adds a notion of state to the system that can be its own logical burden.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8OQT89xYrz6PYhw3flin8mVsaYNhdzQ4ImL9aT2U5msXr5F5NclQR90NR_mABNM0kQ0slYBArAOIchcxeQ1pLliRc7m0JLZznKiajaCHMXpPo4wkatQr97heDPu8Rz_NkbYHtcLFy_8U/s1600/wv09.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1063" data-original-width="1600" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8OQT89xYrz6PYhw3flin8mVsaYNhdzQ4ImL9aT2U5msXr5F5NclQR90NR_mABNM0kQ0slYBArAOIchcxeQ1pLliRc7m0JLZznKiajaCHMXpPo4wkatQr97heDPu8Rz_NkbYHtcLFy_8U/s640/wv09.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Different ways to extend data for the 2/6 DWT operation at the start and end of a vector.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
There are other subtle issues with the DWT in the context of compression. For one, if the data is <i>not</i> structured, the DWT actually increases the storage requirement. Consider all the possible sum and difference outputs for two random 8-bit data points under the Haar DWT. The sum range is [0:510] and the difference range is [-255:255], both seemingly requiring 9-bit results. The signal's entropy hasn't changed; the extra bits are disguising two new redundancies in the data: 1) The sum and difference are either both even or both odd. 2) A larger difference implies a sum closer to 255.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Things get even a bit worse with the 2/6 DWT. If the difference weights are multiplied by 8 to give an integer result, the difference range for six random 8-bit values is [-2550:2550], which would require 13 bits to store. Since there are six inputs per two outputs in the difference operation, it's also harder to see the extra bits as representing some simple redundancies in the transformed data. </div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg73fM9NaBhFz9mrFwYsS2A6h8m_ew7HTQauELPwU7taID_aMg4gtN_NxtX9qEt460N-XcWAQEyHh6y3Rk7v_zXZZqKRMWC8Z872ioKDx9xodb0N0XH50rrgXg5ciugWtuoHmdKxpVS9k4/s1600/wv13.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1007" data-original-width="1600" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg73fM9NaBhFz9mrFwYsS2A6h8m_ew7HTQauELPwU7taID_aMg4gtN_NxtX9qEt460N-XcWAQEyHh6y3Rk7v_zXZZqKRMWC8Z872ioKDx9xodb0N0XH50rrgXg5ciugWtuoHmdKxpVS9k4/s640/wv13.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">65536 random vectors fed through the 2/2 and 2/6 DWT give an idea of the output range.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
The differences in a structured signal will still be concentrated heavily in the smaller values, so compression can still be effective. But it would be nice not to have to reserve so much memory for intermediate results, especially in a multi-stage DWT. Rounding or truncating the data after each sum and difference operation seems justified, since a lossy compression algorithm will wind up discarding least significant bits anyway. But it would be nice to keep the DWT itself lossless, deferring lossy compression to a later operation. In fact, there's an efficient algorithm for performing the DWT operations reversibly without introducing as much redundancy in the form of extra result bits.</div>
<h4 style="clear: both; text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="lifting"></a>The Lifting Scheme</h4>
<div>
What follows are a couple of simple examples of an amazing piece of math that's relatively modern, with the <a href="https://cm-bell-labs.github.io/who/wim/papers/lift2.pdf">original publication</a> in 1995. There's a lot to the theory, but one core concept is that a DWT can be built from smaller steps, each inherently reversible. For example, the Haar (2/2) DWT could be built from two steps as follows:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: center;">
<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV2KWX4eJThUPIQQdLqCKWALhqMkV5kTv5P_iZyzvcYc1HoXot0zJthwDfVK2u_JjhVyYMB4kus3aoukMTN6oJH9rXTT5yHh65hMkLDvgJKm1JfpC2vEP4d5dEejh1ocpjkuw0LKqM4H0/s1600/wv10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1218" data-original-width="1549" height="313" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV2KWX4eJThUPIQQdLqCKWALhqMkV5kTv5P_iZyzvcYc1HoXot0zJthwDfVK2u_JjhVyYMB4kus3aoukMTN6oJH9rXTT5yHh65hMkLDvgJKm1JfpC2vEP4d5dEejh1ocpjkuw0LKqM4H0/s400/wv10.png" width="400" /></a></div>
Included in the forward <span style="color: #ffe599;">Step 2</span> is a truncation of the difference by right shift. This is equivalent to dividing by two and rounding down, and turns the sum output into an integer average, which only requires as many bits as one input. But, remarkably, no information is really discarded. The C code makes the inherent reversibility pretty clear. Even if you truncated more bits of the difference, it would be reversible. In fact, you could skip the sum entirely and just forward x0 to s.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The mechanism of the lifting scheme has effectively eliminated one of the two redundancies that was adding bits to the results of the 2/2 DWT. Specifically, the fact that the sum and difference were either both even or both odd has been traded in for a one-bit reduction in the range of the sum output. Likewise, it can help reduce the range of the difference output of the 2/6 DWT without sacrificing reversibility:</div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYA32Z2NmC27Exp8Js2fqwLBDSx8EJixPOx0xXX4Rh2fjd3dEj9K0fWRhFZ4apPQwH5thJ92E_xImwM7pH9fSsLlV4PzI7HRl7SDZxJSvNuTXt9CVKHCIWRaiiW0GbvR8QVjE-jhWOK4Y/s1600/wv14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="659" data-original-width="1600" height="262" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYA32Z2NmC27Exp8Js2fqwLBDSx8EJixPOx0xXX4Rh2fjd3dEj9K0fWRhFZ4apPQwH5thJ92E_xImwM7pH9fSsLlV4PzI7HRl7SDZxJSvNuTXt9CVKHCIWRaiiW0GbvR8QVjE-jhWOK4Y/s640/wv14.png" width="640" /></a></div>
One additional step is added that calculates the local slope from two adjacent sum outputs. (The symbols z<sup>-1</sup> and z are standard notation for "one sample behind" and "one sample ahead".) After the subtraction, a round-down-bias-compensating constant of two is added in and then the result is right shifted by two, effectively dividing it by four. The result is similar to the four outer 1/8-weighted terms of the six-point difference, but with rounding. However, because this whole step is done identically in the forward and reverse direction, it's still fully reversible.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The calculation of the local slope from two adjacent <i>outputs</i> instead of four adjacent <i>inputs</i> highlights another important efficiency feature of the lifting scheme: intermediate results get reused in later lifting steps. This reduces the total number of shift/add operations (or multiply/add operations, for more complex wavelets). It also means that once the immediate sum and difference steps have been performed on a particular pixel pair, that input pair is no longer needed and its memory can be used to store intermediate results instead. (Fully in-place computation is possible.)</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The 2/6 lifting scheme construction as described above will be the basis for the hardware implementation to follow. One important consideration is that the real implementation must be <a href="https://en.wikipedia.org/wiki/Causal_system">causal</a>, so the "one sample ahead" (z) component of the local slope calculation implies a latency of at least one pixel pair from input to output. This has different consequences for the horizontal DWT, which operates in the same dimension as the sensor readout scan, and the vertical DWT, which must wait for enough completed rows. For this and other reasons, the hardware implementations for each dimension can wind up being quite different.</div>
<h4 style="text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="h26"></a>Horizontal 2/6 DWT Core</h4>
</div>
<div>
For running a multi-stage 2D DWT at 3.8Gpx/s on a Zynq Ultrascale+ SoC, the bottleneck isn't really computation, it's memory access. Writing raw frames to external (PS-side) DDR4 RAM and then doing an in-place 2D DWT on them is infeasible. Even the few Tb/s of BRAM bandwidth available on the PL-side needs to be carefully rationed! For that reason, I decided I wanted the Stage 1 horizontal cores to run directly on the 64 <a href="http://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">pixel input streams</a>, using only distributed memory. Something like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTO8zfheUZKbpGAjudaYxNocjfC059BeSFU2glJyaNED1-Wc0dn7EJNVeAnt6dgDrG2qJTiWIHa7c-yem2iwOL9wp0o0HQlN2f11gsIa04XEcvodHKDrlJGFVYxUIQ2NYeW4IwgEwBvBk/s1600/wv16.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="718" data-original-width="1600" height="284" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTO8zfheUZKbpGAjudaYxNocjfC059BeSFU2glJyaNED1-Wc0dn7EJNVeAnt6dgDrG2qJTiWIHa7c-yem2iwOL9wp0o0HQlN2f11gsIa04XEcvodHKDrlJGFVYxUIQ2NYeW4IwgEwBvBk/s640/wv16.png" width="640" /></a></div>
</div>
<div>
Because the DWT operates on pixel pairs, one register is required to store the even pixel. Then, all the action happens on the odd pixel's clock cycle:</div>
<div>
<ul>
<li>Combinational block A does <span style="color: #76a5af;">Step 1</span> and <span style="color: #ffe599;">Step 2</span> of the 2/6 DWT lifting scheme as described above, placing its results in registers <span style="color: #76a5af;">D_0</span> and <span style="color: #ffe599;">S_0</span>.</li>
<li><span style="color: #ffe599;">S_0</span> and <span style="color: #76a5af;">D_0</span>'s previous values are shifted into <span style="color: #ffe599;">S_1</span> and <span style="color: #76a5af;">D_1</span>.</li>
<li><span style="color: #ffe599;">S_1</span>'s previous value is shifted into <span style="color: yellow;">S_out</span>.</li>
<li>Combinational block B does <span style="color: #c27ba0;">Step 3</span> of the 2/6 DWT lifting scheme, using <span style="color: #ffe599;">S_0</span> and <span style="color: yellow;">S_out</span>'s previous values to compute the local slope and subtract it from <span style="color: #76a5af;">D_1</span>. The result is placed in <span style="color: magenta;">D_out</span>.</li>
</ul>
<div>
This is a tiny core: the seven 16-bit registers get implemented as 112 total Flip-Flops (FFs) and the combinational logic takes around 64 Look-Up Tables (LUTs), as four 16-bit adders. And it <i>has</i> to be tiny, because not only are there 64 pixel input streams, but each stream services two color fields (in the <a href="https://en.wikipedia.org/wiki/Bayer_filter">Bayer-masked</a> color version of the sensor). So that's 128 Stage 1 horizontal cores running in parallel. The good news is that they only need to run at px_clk frequency, 60MHz, which isn't much of a timing burden.</div>
</div>
<div>
<br /></div>
<div>
Unfortunately, that tiny core was a little too good to be true. The 64 pixel input streams from the CMV12000 are divided up into two rows and 32 columns. Within each column, there are only 128 adjacent pixels, and only 64 of a given color field. Remember the part about the 2/6 DWT requiring input extensions of two samples at the beginning and end of stream? Now, each core would need logic to handle that. But unlike the actual beginning and end of a row, in most cases the required data actually does exist, just not in the same stream as the one being processed by the core. A better solution, then, is to tie adjacent cores together with the requisite edge pair data:</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtvZAggxE9bx76j8GYdvMqCqT6NbeTFNS9xlhTYDaGxt4h2RiNCwFfPW6Q_iFN_605H8AAryVU2jZewKFOu2-5H5TmIaPg8JiK9MmGIbnUmIteu01gDcn-y9_qZBlKSRj9V4CQxZcen0g/s1600/wv17.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="586" data-original-width="1380" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtvZAggxE9bx76j8GYdvMqCqT6NbeTFNS9xlhTYDaGxt4h2RiNCwFfPW6Q_iFN_605H8AAryVU2jZewKFOu2-5H5TmIaPg8JiK9MmGIbnUmIteu01gDcn-y9_qZBlKSRj9V4CQxZcen0g/s640/wv17.gif" width="640" /></a></div>
</div>
<div>
Now, the horizontal core can be in one of three states: Normal, during which the core has access to everything it needs from within its own column. <span style="color: #f1c232;">Last Out</span>, where the core needs to borrow the sum of the first pixel pair from the core to its right (S_pp0_fromR). And <span style="color: #f1c232;">First Out</span>, where it needs to borrow the first two sums and the first difference from the core to its right (S_pp0_fromR, S_pp1_fromR, and D_pp0_fromR). The active state is driven by a global pixel counter.<br />
<br />
In addition to the extra switching logic (implemented as LUT-based multiplexers), the cores need to store their first and second pixel pair sums (<span style="color: #ffe599;">S_pp0</span>, <span style="color: #ffe599;">S_pp1</span>) and first pixel pair difference (<span style="color: #76a5af;">D_pp0</span>) for later use by the adjacent core. One more register, <span style="color: #ffe599;">S_2</span>, is also added as a dedicated local sum history to allow the output sum to be the target of one of the multiplexers. The total resource utilization of the Stage 1 horizontal core is 176 FFs (11x16b registers) and 107 LUTs. That's still pretty small, but 128 cores in parallel eats up about 15% of the available logic in the XCZU4.<br />
<br />
Luckily, things get a lot easier in Stage 2 and Stage 3, which only need 32 and 16 horizontal cores, respectively. They're also fed whole pixel pairs from the stage above them, so the X_even register isn't needed. Otherwise, they operate in a similar manner to the Stage 1 core shown above.<br />
<h4>
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="v26"></a>Vertical 2/6 DWT Core</h4>
While I can get away with using only distributed memory for the horizontal cores, the vertical cores need to store several entire rows. This means using Block RAM (BRAM), which is dedicated synchronous memory available on the Zynq Ultrascale+ PL side. The XCZU4 has 128 BRAMs, each with 36Kib of storage. Each color field is 2048px wide, so a single BRAM could store a single row of a single color field (2048 · 16b = 32Kib). I settled on storing eight rows, requiring a total of 32 BRAMs for the four color fields of Stage 1.<br />
<br />
Storing whole rows in each BRAM is the wrong way to do things, though. The data coming from the CMV12000 is primarily parallel by column, and preserving that parallelism for as long as possible is the key to going fast. If all 32 Stage 1 horizontal cores of a given color field had to access a single BRAM every time a new pixel pair was ready at their output (once every four px_clk), it would require eight write accesses per px_clk. While 480MHz is <i>theoretically</i> in-spec for the BRAM, it would make meeting timing constraints much harder. It's much better to divide up the BRAMs into column-parallel groups, each handling 1/8 of the total color field width and receiving data from only four horizontal cores (just a 60MHz write access).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPbOym4LTpUSejjwJc-i137wnO_HKnQeQdoma8ahT560l_0O_ebDgixS7MwYtGLmRNCTkxSFPWAxX873ECKfBedtXvUNhBwrq8s4wbA2qoxQH96LcBcbC3Mkw9am7CWgtu34qOEaQgz9k/s1600/wv18.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="425" data-original-width="1600" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPbOym4LTpUSejjwJc-i137wnO_HKnQeQdoma8ahT560l_0O_ebDgixS7MwYtGLmRNCTkxSFPWAxX873ECKfBedtXvUNhBwrq8s4wbA2qoxQH96LcBcbC3Mkw9am7CWgtu34qOEaQgz9k/s640/wv18.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Stage 1 parallel architecture for a single color field. BRAM writes round-robin through four horizontal cores at 60MHz.</td></tr>
</tbody></table>
Now, each BRAM stores all eight rows for its column group. The vertical 2/6 DWT core can be built around the BRAM, with the DWT operations running on six older rows while two newer rows are being written with horizontal core data. Doing six reads per two writes (180MHz read access) isn't too bad, especially since the BRAM has independent read and write ports. But I can do even better by using a 64-bit read width and processing two pixel pairs at a time in the vertical core. To avoid dealing with a 3/2-speed clock, I decided to implement the reads as six of eight states in a state machine that runs at double px_clk (120MHz).<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3OQh2vJ2twGRAzQAFgDGp_tfohHHwHaaiSX2zNb44rWwDh82SZE-vQe7Yl1utD4sB8mEijLyHe8Ts-NZQlKpac45OzbKStIq8AqS9ubpiUEdoPIrH3B_oCS6bbyh0Wh6PmRXlWG_q0Xo/s1600/wv19.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="773" data-original-width="1430" height="344" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3OQh2vJ2twGRAzQAFgDGp_tfohHHwHaaiSX2zNb44rWwDh82SZE-vQe7Yl1utD4sB8mEijLyHe8Ts-NZQlKpac45OzbKStIq8AqS9ubpiUEdoPIrH3B_oCS6bbyh0Wh6PmRXlWG_q0Xo/s640/wv19.gif" width="640" /></a></div>
<div style="text-align: left;">
Since it has access to all the data it needs in the six oldest rows, the vertical core <i>can</i> actually be pretty simple. Input extensions are only needed at the top and bottom of a frame (sort-of, see below); no data is needed from adjacent cores. It's not as computationally efficient as the horizontal core: <span style="color: #76a5af;">Step 1</span> and <span style="color: #ffe599;">Step 2</span> are repeated (via combinational block A) three times on a given row pair as it gets pushed through. This is done intentionally to avoid having to write back local sums to the BRAM, so the vertical core only needs read access.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Since the vertical core operates on 64-bit registers (two pixel pairs at a time), all the multiplexers and adders are four times as wide, giving a total LUT count of 422, roughly four times that of the horizontal core. This seems justified, with a ratio of 4:1 for horizontal:vertical cores in Stage 1. Still, it means the complete Stage 1 uses about 30% of the total logic available on the XCZU4. I do get a bit of a break on the FF count, since this core only has six 64-bit registers (384 FFs, much less than four times the horizontal core FF count).</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The output of the vertical core is two 64-bit registers containing the vertical sum and difference outputs of two pixel pairs. Because of the way the horizontal sums and differences are interleaved in the BRAM rows, this handily creates four 32-bit pixel pairs to route to either the next wavelet stage (for the LLx pair) or the compression hardware (for the HLx, LHx, and HHx pairs). The LLx pair becomes the input for the next stage horizontal cores.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvmwjTxyMNWEpmZxjHb8MDCUpwHBbXyKrRH8CzR_cIl5zB-dmv7bvyt68AUA1SkohPhLTQp5bdzn0xbnhwGoqpd2SvTJBuGSJCYPa2Spr_8zr-r6XP7IReYaqyTy30L_FYBW7tB3yXMaM/s1600/wv20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="497" data-original-width="1600" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvmwjTxyMNWEpmZxjHb8MDCUpwHBbXyKrRH8CzR_cIl5zB-dmv7bvyt68AUA1SkohPhLTQp5bdzn0xbnhwGoqpd2SvTJBuGSJCYPa2Spr_8zr-r6XP7IReYaqyTy30L_FYBW7tB3yXMaM/s640/wv20.png" width="640" /></a></div>
<div style="text-align: left;">
At each stage, the color field halves in width and height. But, the vertical cores need to store eight rows at all stages, so only the width reduction comes into play. Thus, Stage 2 needs 16 vertical cores (four per color field) and Stage 3 needs eight (two per color field). The extra factor of two is not lost completely, though: at each stage, the write and read access rates of the vertical core BRAM are also halved. In total, the three-stage architecture looks something like this:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLLq7B5l50lNHRrRx38_xELqcUQM7rCe11nzTId_1W57ddYs4R2VuSrhji79iyEZxN9WIiiEEGqLGIZL5JFhzXHE_Pc99dBCfEWUFcBEivBsa9opQWiNjIRNsFDJ5A3RZ50l5NBV75VSs/s1600/wv21.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1414" data-original-width="1600" height="564" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLLq7B5l50lNHRrRx38_xELqcUQM7rCe11nzTId_1W57ddYs4R2VuSrhji79iyEZxN9WIiiEEGqLGIZL5JFhzXHE_Pc99dBCfEWUFcBEivBsa9opQWiNjIRNsFDJ5A3RZ50l5NBV75VSs/s640/wv21.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Three-stage DWT architecture for a single color field. In total, 44 horizontal cores and 14 vertical cores are used per color field. Data rates are divided by four at each stage, since it only processes the LLx output from the previous stage.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
Vertical core outputs that don't go to a later stage (HHx, HLx, LHx, and LL3) are sent to the compression hardware, which is where we're heading next as well, after tying up some loose ends.</div>
<h4 style="text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="extensions"></a>Input Extensions?</h4>
<div>
I mentioned that the horizontal cores are chained together so that the DWTs can access adjacent data as needed within a row, but what happens with the left-most and right-most cores, at the beginning and end of a row? And what happens at the start and end of a frame, in the vertical cores? Essentially, nothing:</div>
</div>
<div>
<ul>
<li>For the horizontal DWT, the left-most and right-most cores are just tied to each other as if they were adjacent, a free way to implement periodic extension.</li>
<li>For the vertical DWT, the BRAM contents are just allowed to roll over between frames. The first rows of frame N reference the last rows of frame N-1.</li>
</ul>
<div>
This isn't necessarily the optimal approach for an image; symmetric extension will produce smaller differences, which are easier to compress. But, it's fast and cheap. It's also reversible if the inverse DWT implements the same type of extension of its sum inputs. Even the first rows of the first frame can be recovered if the BRAM is known to have started empty (free zero padding).<br />
<h4>
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="quantizer"></a>Quantizer</h4>
</div>
</div>
<div>
If you take a typical raw image straight off a camera sensor, run it through a multi-stage 2D DWT, and then zip the result, you'll probably get a compression ratio of around 3:1. This isn't as much a metric of the compression method itself as of the image, or maybe even of our brains. We consider 50dB to be a good signal-to-noise ratio for an image. No surprise, this gives an RMS noise of just under 1.0 in the standard 8-bit display depth of a single color channel. But a 10- or 12-bit sensor will probably have single-digit noise, which winds up on the DWT difference outputs. This single-digit noise is distributed randomly on the frame and <a href="https://en.wikipedia.org/wiki/Entropy_(information_theory)">requires</a> a few bits per pixel to encode, limiting the lossless compression ratio to around 3:1.<br />
<br />
To get to 5:1 or more, it's necessary to introduce a lossy stage in the form of re-quantization to a lower bit depth. It's lossy in the sense of being mathematically irreversible, unlike the quantization in the lifting scheme. Information will be discarded, but we can be selective about what to discard. The goal is to reduce the entropy with as little effect on the subjective visual quality of the image as possible. Discarding bits from the difference channels, especially the highest-frequency ones (HH1, HL1, LH1), has the least visual impact on the final result. This is especially true if the least significant bits of those channels are lost in sensor noise anyway.<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgw7RnrPkPyd7B8mbalDRFs1iYE4CdjT_DHmdj29gdAm7p3A9sTlIeThezUoxxWpSMKxuk7GOBsxyZOiPM2ld0NrTYjGwGhGF-jvyn3E5J054vXiDv97n0ND3jzcnia2B0E_jh8KmpY7nA/s1600/wv03.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="187" data-original-width="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgw7RnrPkPyd7B8mbalDRFs1iYE4CdjT_DHmdj29gdAm7p3A9sTlIeThezUoxxWpSMKxuk7GOBsxyZOiPM2ld0NrTYjGwGhGF-jvyn3E5J054vXiDv97n0ND3jzcnia2B0E_jh8KmpY7nA/s1600/wv03.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bits can be discarded from the difference channels with far less visual impact than discarding bits from the average.</td></tr>
</tbody></table>
<div style="text-align: center;">
<div style="text-align: left;">
In a sense, the whole purpose of doing the multi-stage DWT was to separate out the information content into sub-bands that can be prioritized by visual importance:</div>
<div style="text-align: left;">
</div>
<ol>
<li style="text-align: left;">LL3. This is kept as raw 10- or 12-bit sensor data.</li>
<li style="text-align: left;">LH3, HL3, and HH3.</li>
<li style="text-align: left;">LH2, HL2, and HH2.</li>
<li style="text-align: left;">LH1, HL1, and HH1.</li>
</ol>
<div style="text-align: left;">
Arguably, the HHx sub-bands are of lower priority than the LHx and HLx sub-bands on the same level. This would prioritize the fidelity of horizontal and vertical edges over diagonal ones. With these priorities in mind, each sub-band can be configured for a different level of re-quantization in order to achieve a target compression ratio.</div>
<div>
<br /></div>
<div>
<div style="text-align: left;">
In C, the fastest way to implement a configurable quantization step would be a variable right shift, causing the signal to be divided by {2, 4, 8, ...} and then rounded down. But implementing a <a href="https://en.wikipedia.org/wiki/Barrel_shifter">barrel shifter</a> in LUT-based multiplexers eats resources quickly if you need to process a bunch of 16-bit variable right shifts in parallel. Fortunately, there are dedicated multipliers distributed around the PL-side of the Zynq Ultrascale+ SoC that can facilitate this task. These DSP slices have a 27-bit by 18-bit multiplier with a full 45-bit product register. To implement a configurable quantizer, the input can be multiplied by a software-set value and the output can be the product, right shifted by a <i>constant</i> number of bits (just a bit-select):<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKWi2SiTMmBcn0a_mvICmwyEZrx77kNYPpO_tlMaJvNW1I9IeRL2Tav9fhTRQzLcH7Pt677tTSdMQp9P_p-E7LsyCqnV-6iQWUNCE9c0pWBLc4RY0qaLHEQUkXKHML_Bz_JSiGXcocPWU/s1600/wv22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="479" data-original-width="1136" height="167" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKWi2SiTMmBcn0a_mvICmwyEZrx77kNYPpO_tlMaJvNW1I9IeRL2Tav9fhTRQzLcH7Pt677tTSdMQp9P_p-E7LsyCqnV-6iQWUNCE9c0pWBLc4RY0qaLHEQUkXKHML_Bz_JSiGXcocPWU/s400/wv22.png" width="400" /></a></div>
This is more flexible than a variable shift, since the multiplier can be any integer value. Effectively, it opens up division by non powers-of-two. For example, 85/256 for an approximate divide-by-3. The DSP slices also have post-multiply logic that can be used, among many other things, to implement different rounding strategies. An unbiased round-toward-zero can be implemented by adding copies of the input's sign bit to the product up to (but not including) the first output bit:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAWRMGeuazCAii5N4Mg10-9k8rqjPkDROxjxpGzHLmFHYTH7v203ULV5k90q6luIbvF39LTLjHCC0RRRr4DSz6Y3VK2WO896GCYfuy6C8Pq2Sh46fCi-AOsEyB9NE3cYyLccHfqqIThXc/s1600/wv23.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="1228" height="217" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAWRMGeuazCAii5N4Mg10-9k8rqjPkDROxjxpGzHLmFHYTH7v203ULV5k90q6luIbvF39LTLjHCC0RRRr4DSz6Y3VK2WO896GCYfuy6C8Pq2Sh46fCi-AOsEyB9NE3cYyLccHfqqIThXc/s400/wv23.png" width="400" /></a></div>
The XCZU4 has 728 DSP slices distributed throughout its PL-side, so dedicating a bunch of these to the quantization task seems appropriate. The combined output of all the wavelet stages is 64 16-bit values per px_clk, so 64 DSPs would do the trick with very little timing burden. But there's a small catch that has to do with how the final quantized values get encoded.</div>
</div>
<div>
<h4 style="text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="encoder"></a>Variable-Length Encoder</h4>
</div>
<div>
<div style="text-align: left;">
So far we've done a terrible job at reducing the bit rate: the sensor's 37.75Gb/s (4096 · 3072 · 10b · 300fps) has turned into 61.44Gb/s at the quantizer (64 · 16b · 60MHz). But, if the DWT and quantizer have worked, most of the 16-bit values in HHx, HLx, and LHx should be zeros. A lot will be ±1, fewer will be ±2, fewer ±3, and so on. There will be patches of larger numbers for actual edges, but the probability will be heavily skewed toward smaller values.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigLB4VApJPEkQF5_iM2sk9lHG_jxi_yCL_LmOg3CQxfXYm8hwn6KYt9CFg1La3ME3f2xQ6x_IZ2VddsApRL2gidQdSC3m91is4Ihs4qq0WYE4YQNzg6fogcEoFyHEa_x2yetraoANGd8M/s1600/wv24.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigLB4VApJPEkQF5_iM2sk9lHG_jxi_yCL_LmOg3CQxfXYm8hwn6KYt9CFg1La3ME3f2xQ6x_IZ2VddsApRL2gidQdSC3m91is4Ihs4qq0WYE4YQNzg6fogcEoFyHEa_x2yetraoANGd8M/s640/wv24.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Probability distribution for a typical set of DWT high-frequency outputs. Green indicates zero difference outputs.</td></tr>
</tbody></table>
</div>
</div>
<div>
<div style="text-align: left;">
If the new probability distribution has an <a href="https://en.wikipedia.org/wiki/Entropy_(information_theory)">entropy</a> of less than 2.00 bits, it's theoretically possible to achieve 5:1 compression. One way to do this is with a variable-length code, which maps inputs with higher probability to outputs with fewer bits. A known or measured probability distribution of the inputs can be used to create the an efficient code, such as in <a href="https://en.wikipedia.org/wiki/Huffman_coding">Huffman coding</a>. Adapting the code as the probability distribution changes is a little beyond the amount of logic I want to add at the moment, so I will just take an educated guess at a fixed code that will work okay given the expected distributions:</div>
<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5Am3OHuC1cppxgWHui5UQ-HR_WBkdieg06wHVBanddxn-Sn3gAToA9ds-bLixWOfpbIAoiWlflmdOe45E7z1lsyznn92Atz6Uu-ghP3MHyJNfVTsUarRFVhxaNJe5GQ11w9mExmJ7ObE/s1600/wv25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1009" data-original-width="1600" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5Am3OHuC1cppxgWHui5UQ-HR_WBkdieg06wHVBanddxn-Sn3gAToA9ds-bLixWOfpbIAoiWlflmdOe45E7z1lsyznn92Atz6Uu-ghP3MHyJNfVTsUarRFVhxaNJe5GQ11w9mExmJ7ObE/s640/wv25.png" width="640" /></a></div>
This encoder looks at four <i>sequential</i> quantized values and determines how many bits are required to store the largest of the four. It then encodes that bit requirement in a prefix and concatenates the reduced-width codes for each of the four values after that. This is all relatively fast and easy to do in discrete logic. Determining how many bits are required to store a value is similar to the <a href="https://en.wikipedia.org/wiki/Find_first_set">find first set</a> bit operation. Some bit bins are grouped to reduce multiplexer count for constructing the coded output. Applying this code to the three example probability distributions above gives okay results:<br />
<br />
<ul>
<li><span style="color: #b45f06;">LH1:</span> 1.38bpp (7.26:1) compared to 1.14bpp (8.77:1) theoretical.</li>
<li><span style="color: yellow;">HL1:</span> 1.39bpp (7.20:1) compared to 1.06bpp (9.43:1) theoretical.</li>
<li><span style="color: magenta;">HH1:</span> 2.41bpp (4.16:1) compared to 1.84bpp (5.44:1) theoretical.</li>
</ul>
<div>
To get closer to the theoretical maximum compression, more logic can be added to revise the code based on observed probabilities. It might also be possible to squeeze out more efficiency by using local context to condition the encoder, almost like an extra mini wavelet stage. But since this has to go really fast, I'm willing to trade compression efficiency for simplicity for now.</div>
<div>
<br /></div>
<div>
I emphasized that the four quantized values need to be <i>sequential</i>, i.e. from a single continuous row of data. The probability distribution depends on this, and the parallelization strategy of the quantizers and encoders must enforce it. The vertical core outputs are all non-adjacent pixel pairs, either from different levels of the DWT, different color fields within a level, or different columns within a color field. So, while a single four-pixel quantizer/encoder input can round-robin through a number of vertical core outputs, the interface must store one previous pixel pair. I decided to group them in 128-bit compressor cores, each with two 4x16b quantizers and encoders:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixs0lhRuvtjU5TjYl6PbtyzS41zAcG0WirVPlXT8KZww1YY-6rHP4whbNU6oStCWzfkI3FgwPAXyruVTqYApL7t7Oq-DdVGppU-2SxyeS-IRZ5StK9t1Gb0DaWhdOJDE0D9PAS3DuPu5c/s1600/wv26.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="867" data-original-width="1600" height="346" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixs0lhRuvtjU5TjYl6PbtyzS41zAcG0WirVPlXT8KZww1YY-6rHP4whbNU6oStCWzfkI3FgwPAXyruVTqYApL7t7Oq-DdVGppU-2SxyeS-IRZ5StK9t1Gb0DaWhdOJDE0D9PAS3DuPu5c/s640/wv26.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Each input is a 32-bit pixel pair from a single vertical core output. During the "even" phase, previous pixel pair values are saved in the in_1[n] registers. During the "odd" phase, the quantizers and encoders round-robin through the inputs, encoding the current and previous pixel pairs of each. A final step merges the two codes and repacks them into a 192-bit buffer. When this fills past 128 bits, a write is triggered to a 128-bit FIFO (made with two BRAMs) that buffers data for DDR4 RAM writing, using a similar process as I previously used for <a href="http://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">raw sensor read-in</a>.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The eight-input compressor operates on two pixel pairs simultaneously at each px_clk, covering eight inputs in four px_clk. This matches up well to the eight vertical cores per color field of Stage 1, which update their outputs every fourth px_clk. In total, Stage 1 uses 12 compressor cores: four color fields each for HH1, HL1, and LH1 pixel pair outputs. Things get a little more confusing in Stage 2 and Stage 3, where the outputs are updated less frequently. To balance the load before it hits the RAM writer, it makes sense to increase the round-robin by 2x at each stage. When load-balanced, the overall mapping uses winds up using 16 compressor cores:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9inwvuqUJ_-sw-MSA8vjX6HDJ3nC7qhccFdXZ4TBsENduzCZAf7-nkUaTy3NzjBe6CYd0qFsTNFlc0Ft024FOqBPh2kiURzlq3GljFfvy3Ze661iqwUmYyX1FZrd08Vp1X37KqWC2cOY/s1600/wv27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1170" data-original-width="1600" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9inwvuqUJ_-sw-MSA8vjX6HDJ3nC7qhccFdXZ4TBsENduzCZAf7-nkUaTy3NzjBe6CYd0qFsTNFlc0Ft024FOqBPh2kiURzlq3GljFfvy3Ze661iqwUmYyX1FZrd08Vp1X37KqWC2cOY/s640/wv27.png" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Each core handles 1/16 of the 61.44Gb/s output from the DWT stages, writing compressed data into its BRAM FIFO. An AXI Master module checks the fill level of each FIFO and triggers burst writes to PS DDR4 RAM as needed. A single PL-PS AXI interface has a maximum theoretical bandwidth of 32Gb/s at 250MHz, which is why I needed two of them in parallel for <a href="http://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">writing raw images to RAM</a>. But since the target compression ratio is 5:1, I should be able to get away with a single AXI Master here. In that case, the BRAM FIFOs are critical for handling transient entropy bursts.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
And that's it, we've reached the end of this subsystem. The 128b-wide AXI interface to the PS-side DDR controller is the final output of the wavelet compressor. What becomes of the data after that is a <a href="http://scolton.blogspot.com/2019/07/benchmarking-nvme-through-zynq.html">problem for another day</a>.</div>
<h4 style="text-align: left;">
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="wrap"></a>Wrapping Up</h4>
<div>
Even though most of the cores described above are tiny, there are just so many of them running in parallel that the combination of <a href="http://scolton.blogspot.com/2019/09/cmv12000-full-speed-384gbs-read-in-on.html">CMV12000 Input stage</a> and Wavelet stages described here already takes up 70% of the XCZU4's LUTs, 39% of its FFs, and 69% of its BRAMs.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBOd8HtI-gfA6MhyphenhyphenBue9snKnDnwwmxy2yelC2m3PtAWuKSW8lHG-FaSLAxv-F6izTK2wkOWqvKfyY3HZL81hpCjWWNHMcVsN4QvpH-EYlvr6VU8nYofcFfOmln-SxIZF4fZv2hqEE9j04/s1600/wv28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1193" data-original-width="1546" height="491" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBOd8HtI-gfA6MhyphenhyphenBue9snKnDnwwmxy2yelC2m3PtAWuKSW8lHG-FaSLAxv-F6izTK2wkOWqvKfyY3HZL81hpCjWWNHMcVsN4QvpH-EYlvr6VU8nYofcFfOmln-SxIZF4fZv2hqEE9j04/s640/wv28.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div>
I'm sure there's still some room for optimization. Many of the modules are running well below their theoretical maximum frequencies, so there's a chance that some more time multiplexing of certain cores could be worthwhile. But I think I would rather stick to having many slow cores in parallel. The next step up would probably be the <a href="https://shop.trenz-electronic.de/en/TE0808-04-6BE21-A-UltraSOM-MPSoC-Module-with-Zynq-UltraScale-XCZU6EG-1FFVC900E-4-GB-DDR4">XCZU6</a>, which has more than double the logic resources. It's getting more expensive than I'd like, but I will probably need it to add more pieces:</div>
<div>
<ul>
<li>The PCIe bridge and, if needed, NVMe hardware acceleration.</li>
<li>Decoding and preview hardware, such as HDMI output generation. This can be much less parallel, since it only needs to run at 30fps. Maybe the ARMs can help.</li>
<li>128 <i>more</i> Stage 1 horizontal cores to deal with the sensor's 2048x1536 sub-sampling mode, where four whole rows are read in at once. This should run at above 1000fps.</li>
</ul>
<div>
For now, though, let's run this machine:<br />
<br />
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/ep0a7_K0EIs" width="640"></iframe>
</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br />
That's a quick test at 4096x2304, with all the quantizers (other than LL3) set to 32/256 and the frame rate maxed out (400fps at 16:9). This results in an overall compression ratio of about 6.4:1 for this clip. The second half of the clip shows the wavelet output at different stages, although the sensor noise wreaks havoc on the H.264. It's hard to tell anything from the YouTube render, but here's a PNG frame (<a href="https://scolton-www.s3.amazonaws.com/video/wv33.png">full size</a>):<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1IpOjL06e6vP6QlSIJC4QRYeRxnTu8yQgw87S5rAJsPteQJ5Z1sk0OdCMpKJc5vrU2TxOPS97JRNjJpmb5qGFt1fnZo6__hrB6eyzaJBjnoRQaFjjfil9o77_bpZliPKJNfcUlfGZugI/s1600/wv33.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1IpOjL06e6vP6QlSIJC4QRYeRxnTu8yQgw87S5rAJsPteQJ5Z1sk0OdCMpKJc5vrU2TxOPS97JRNjJpmb5qGFt1fnZo6__hrB6eyzaJBjnoRQaFjjfil9o77_bpZliPKJNfcUlfGZugI/s640/wv33.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">No Kerbals were harmed in the making of this video.</td></tr>
</tbody></table>
<div>
The areas of high contrast look pretty good. I'm happy with the reconstruction of the text and QR code on the label. The smoke looks fine - it's mostly out of focus anyway. Wavelet compression is entirely intraframe, so things like smoke and fire don't produce any motion artifacts. There are a few places where the wavelet artifacts do show up, mostly in areas with contrast between two relatively dark features. For example on the color cube string:<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRmrgoJ49SuJ8dXyy9CDPWIBCqE5fmuYbqGg3qCQmZnFNcMapAlU7JexsK6nLBg5uJaKJtKJ1BYjyjl5XDtTVUxLiaWJa61MewoIkvFxghpE4ztKxnRZ6m4YxJgI8j_c8Ubw8PrwjPxWY/s1600/wv30.png" imageanchor="1"><img border="0" data-original-height="1316" data-original-width="1587" height="530" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRmrgoJ49SuJ8dXyy9CDPWIBCqE5fmuYbqGg3qCQmZnFNcMapAlU7JexsK6nLBg5uJaKJtKJ1BYjyjl5XDtTVUxLiaWJa61MewoIkvFxghpE4ztKxnRZ6m4YxJgI8j_c8Ubw8PrwjPxWY/s640/wv30.png" width="640" /></a></div>
<div style="text-align: left;">
<br />
Probably, HH3, HL3, and LH3 should get back one or two bits (quantizer setting of 64-128). On other hand, HH1 might be able to go down one more bit since it looks like it's still encoding mostly sensor noise. I'm not sure how much the quantizer settings will need to change from scene to scene, or even from frame to frame, but overall I think it'll be easy to maintain good image quality at a target ratio of 5:1. I also have a few possibilities for improving the compression efficiency without adding much more logic, such as adding some local context-based prediction to the quantizer DSP's adder port.</div>
<div style="text-align: left;">
<br />
I'll probably take a break from wavelets for now, though, and return to the last big component of this system: the NVMe SSD write. Now that the data is below 1GB/s, I need a place to put it. The RAM will still be used as a big FIFO frame buffer, but ultimately the goal is to continuously record. I also want to drop in a color sensor at some point, since this wavelet compression architecture is really meant for that (four independent color fields). Glad to have the most unknown subsystem completed though!<br />
<br />
That's way too much information for a single post, but oh well. I'll just end with some wavelet art from the incomplete last frame of the sequence above.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQlHNRZ5uFpEIzMpvykFg0ccJcvOUL6ktEEYmLBZMMWMY6-1mHHG_2JqNSIlTE-rGh4mAYi7OLse7lq4w9IXTA0X1hdh1S-XB2zmHiWvD24EbHunPmKhSIIQCQio27LJJ703ibgO0zx_w/s1600/360.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQlHNRZ5uFpEIzMpvykFg0ccJcvOUL6ktEEYmLBZMMWMY6-1mHHG_2JqNSIlTE-rGh4mAYi7OLse7lq4w9IXTA0X1hdh1S-XB2zmHiWvD24EbHunPmKhSIIQCQio27LJJ703ibgO0zx_w/s640/360.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">It's interesting to see what uninitialized RAM looks like when run through the decoder and inverse DWT.</td></tr>
</tbody></table>
<h4>
<a href="https://scolton.blogspot.com/2019/10/real-time-wavelet-compression-for-high.html" name="info"></a>More Information</h4>
<div>
I put the Verilog source for the modules described above <a href="https://github.com/coltonshane/WAVE-Vivado">here</a>. There isn't enough there to clone a working project from scratch, but you can take a look at the <a href="https://github.com/coltonshane/WAVE-Vivado/tree/master/base_cmd.srcs/sources_1/ip">individual cores</a>. Keep in mind that these are just my implementations and I am far from a Verilog master. I would love to see how somebody with 10,000 hours of Verilog experience would write the modules described above.</div>
<div>
<br /></div>
<div>
Here are some unordered references to other good wavelet compression resources:</div>
<div>
<ul>
<li>The <a href="https://github.com/gopro/cineform-sdk">GoPro CineForm SDK</a> and <a href="http://cineform.blogspot.com/">Insider Blog</a> has a ton of good discussion of wavelet compression, including a history of the CineForm codec going back to 2005.</li>
<li>This paper, by some of the big names, lays the foundation for reversible integer-to-integer wavelet transforms, including the one described above: R. Calderbank, I. Daubechies, W. Sweldens, and B.-L. Yeo, “<a href="https://www.sciencedirect.com/science/article/pii/S1063520397902384">Wavelet Transforms That Map Integers to Integers</a>,” <i>Applied and Computational Harmonic Analysis</i>, vol. 5, pp. 332-369, 1998.</li>
<li>The <a href="http://wavelets.pybytes.com/">Wavelet Browser</a>, by PyWavelets, has a catalog of wavelets to look at. Interestingly, what I've been calling the 2/6 wavelet is there called the <a href="http://wavelets.pybytes.com/wavelet/rbio1.3/">reverse biorthogonal 1.3</a> wavelet.</li>
<li>This <a href="http://www.cs.tut.fi/~tabus/course/SC/246pagesCourseonJPEG2000.pdf">huge set of course notes on JPEG2000</a>, which is also wavelet-based, has a lot of information on the 5/3 and 9/7 wavelets used there, as well as quantization and encoding strategies.</li>
</ul>
</div>
</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com2tag:blogger.com,1999:blog-8200098102909041178.post-48714731144985293872019-10-05T20:56:00.003-04:002019-10-06T10:19:58.344-04:00KSP: Laythe Colony Part 4, Drop Ships and Lonely RoversAfter the <a href="http://scolton.blogspot.com/2019/05/ksp-laythe-colony-part-3-colony-ships.html">second Jool launch window</a>, I still had 196 days to get a few extra ships off Kerbin before its <a href="http://scolton.blogspot.com/2017/08/ksp-laythe-colony-part-1.html">destruction</a> on Year 3, Day 0. They couldn't transfer to Jool until the third launch window - around Year 3, Day 260 - but they could still get out of harm's way. I hadn't specified exactly <i>how</i> Kerbin is destroyed, but since this entire scenario is based on <a href="https://www.goodreads.com/book/show/22816087-seveneves">Seveneves</a>, I think it was reasonable to say that these ships should not sit in cismunar orbit. So I decided to send them out to Minmus for parking.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghhfg9FndKweshyek5xmWIQgDmXFt6ZEtsNfeaDxK5sK-XcJ8Z4gjPfcP5myrIXz_6t5PLnwJNBAi81vnPVkxbO27KVRPEBD9IsHf6uOqUcAqJSoTmEHS0hhounQur8Ncd4CXV30kf1sg/s1600/ld30.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghhfg9FndKweshyek5xmWIQgDmXFt6ZEtsNfeaDxK5sK-XcJ8Z4gjPfcP5myrIXz_6t5PLnwJNBAi81vnPVkxbO27KVRPEBD9IsHf6uOqUcAqJSoTmEHS0hhounQur8Ncd4CXV30kf1sg/s640/ld30.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Colony ship #11 or #12 - I lost count. Parked at Minmus for a front row seat to the end of the world.</td></tr>
</tbody></table>
<div style="text-align: left;">
By this point I was getting pretty tired of building colony ships. Each one takes about a dozen launches to assemble, crew, and fuel in low Kerbin orbit. But I managed to get two more built and parked at Minmus. I also realized that there would be a little bit of a housing shortage on Laythe with the extra 72 Kerbals these colony ships carry, so I sent up one more <a href="http://scolton.blogspot.com/2018/11/ksp-laythe-colony-part-2-robotic-fleet.html">HAB1</a> transfer ship as well. But parking ships in Minmus orbit isn't exactly efficient, and I am running a pretty tight Δv budget. A perfect opportunity, then, to create one last piece of hardware for this mission.</div>
<h4 style="text-align: left;">
The Drop Ships</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzV7mAMW1ddXxqmhMtY7VJQl5qvCJ8-TmfFOvfl9v31WfRzeSSoXe6dltSk0ORTPoEpG2yJfg86v9vS7FZE2hAR_vZ_DZiJELrun3KCBPM4Sh-9krP2x_wHQgN1BBgzF1JUtlgREf-Khw/s1600/ld23_crop.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="852" data-original-width="1600" height="340" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzV7mAMW1ddXxqmhMtY7VJQl5qvCJ8-TmfFOvfl9v31WfRzeSSoXe6dltSk0ORTPoEpG2yJfg86v9vS7FZE2hAR_vZ_DZiJELrun3KCBPM4Sh-9krP2x_wHQgN1BBgzF1JUtlgREf-Khw/s640/ld23_crop.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The DS1 lander, a last-minute mining platform and fuel tanker for the fleet.</td></tr>
</tbody></table>
<div style="text-align: left;">
Until now, the only ships in my fleet with mining capabilities were the <a href="http://scolton.blogspot.com/2018/11/ksp-laythe-colony-part-2-robotic-fleet.html">LR1</a> rovers, which can refuel space planes on the surface of Laythe. The planes can then climb into Laythe orbit and transfer any spare fuel to the colony ships. But it would take quite a few launches to fully refuel the colony ships this way. Better to mine on a moon with a shallow gravity well, like Pol, and net a bunch more fuel. So I designed a drop ship mining platform/tanker to do just that. Refueling the ships parked at Minmus before the third Jool transfer window would be a good test.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I've done space planes and straightforward powered descent, but never a true VTOL in the sense of a ship that is designed to hover and translate horizontally looking for flat ground or good mining prospects. Most of my knowledge about drop ships comes from watching <a href="https://www.youtube.com/channel/UCqgsE2C9kx62D1zYWHyAEow">Cupcake Landers</a> videos. I just tried to make it symmetric, place the C.G. properly, and set up the fuel tanks so that the C.G. doesn't shift much as they drain.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijIIPp35bs3Uq1O8ET06HH63o5DRP9Hrt1ctLXgA3x_WvFoPALKKvMdZZsUPHyPDmw3kDp4OQtqFZIAH6U_K6k3_PNcb4AghSlqKyQ9xOKwhrPhltZnrnZM_ecFD6BDS5tJLZfbWo8XKk/s1600/ld20.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijIIPp35bs3Uq1O8ET06HH63o5DRP9Hrt1ctLXgA3x_WvFoPALKKvMdZZsUPHyPDmw3kDp4OQtqFZIAH6U_K6k3_PNcb4AghSlqKyQ9xOKwhrPhltZnrnZM_ecFD6BDS5tJLZfbWo8XKk/s640/ld20.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Drop ship mining practice on Minmus.</td></tr>
</tbody></table>
<div style="text-align: left;">
Even though they're essentially flying fuel tanks, cruising through mountain ranges in the low gravity of Minmus in them is easy and actually kind-of fun. Normally I'm trying to time suicide burns just right or not stall out my space planes, both of which are more stressful technical tasks. Piloting a drop ship is closer to a <a href="https://www.youtube.com/watch?v=MllGbvFBP2k">sci-fi</a> <a href="https://www.youtube.com/watch?v=8EtnyKXacF0">landing</a> <a href="https://www.youtube.com/watch?v=rNpbZdEH2M4">experience</a>. Which reminds me: if you're looking for a quick diversion from the brutally technical challenge of KSP, <a href="http://outerwilds.com/">Outer Wilds</a> is a beautiful (and creepy) exploration/mystery game with some incredible open-world storytelling. Absolutely worth going in blind and playing through.<br />
<br />
Back to Minmus mining, though. I only had time to build two of these drop ships. They can operate autonomously, but they also have room for a pilot, for navigation in frontier areas with poor relay coverage, and an engineer, for more efficient mining. I realized while building these final few ships that I neglected to put relay antennas on the <a href="https://scolton.blogspot.com/2019/05/ksp-laythe-colony-part-3-colony-ships.html">colony ships</a>, something that is required for remote piloting rovers, space planes, or drop ships. Since I might need to do a lot of remote piloting in the Jool system, I decided to steal a couple relay satellites from Kerbin orbit.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitlk_bYJlL76pOQ-NJVhkJtF0ZPKIuz6VB1zQ064fmUkmfizUJzDKOBQQ7mIjgi2OWKrIwx7AES5p4AdwpcZMPOMRTeHPMrG2JKWmLu-bR0xZx58AN2fTcLQ3HPLkw0cs9g69QX7bXoSs/s1600/ld33.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1074" data-original-width="1600" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitlk_bYJlL76pOQ-NJVhkJtF0ZPKIuz6VB1zQ064fmUkmfizUJzDKOBQQ7mIjgi2OWKrIwx7AES5p4AdwpcZMPOMRTeHPMrG2JKWmLu-bR0xZx58AN2fTcLQ3HPLkw0cs9g69QX7bXoSs/s640/ld33.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Stealing a satellite with the grabby claw I knew would come in handy.</td></tr>
</tbody></table>
<div style="text-align: left;">
Once these ships leave Kerbin orbit, there won't be any need for a Kerbin <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2hOdKD1UWiovwfyp38DNwR89tdp3Nxa3kNh2_PfY6UNRl6G2CKifiX2TeFg1b7Mpp_JBclb3M2psc0psmVdpcb-QsFjFEZ2ayQcToAXwVJteD_e6sJK_Rd5LQcyV6l9eHBdLJBmTQA1o/s1600/lc00.png">comms network</a> anymore, so I (literally) grabbed some of Kerbin's relay satellites with the last two colony ships. It is possible to create a remote piloting connection through a relay satellite in a grabby claw, something I find satisfyingly appropriate for Kerbal-style mission "planning". Anyway, I made a few round trips to Minmus surface to refuel the ships of the third wave and then that was it for Kerbin.</div>
<h4 style="text-align: left;">
<span style="color: red;">0 Days Remain</span></h4>
<div>
On Year 3, Day 0, time was up for the Kerbal home planet. The remaining population (of 432 Kerbals) was in flight, either on the way to Jool or at Minmus awaiting the third transfer window. No more hardware would be launched and the roughly four kilotons of ship and propellant in the fleet would have to become the Laythe colony. But it would still be almost another two years before the first colony ship arrives in the Jool system. Before that, the <a href="http://scolton.blogspot.com/2018/11/ksp-laythe-colony-part-2-robotic-fleet.html">robotic fleet</a> would have to lay the groundwork.</div>
<h4>
The Lonely Rovers</h4>
</div>
<div style="text-align: left;">
The 18 ships of the first Jool launch window arrived at their destination during the second half of Year 3. I set up the transfers such that the relay satellites would arrive first, since having a working comms net in the Jool system would be crucial to the rest of the mission. The RS3 ships and especially the ion engine satellites themselves have plenty of Δv to spare, so I just brute-forced them into useful coverage orbits around Jool and Laythe.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg21ZEICOelps5ws9ys2wSALrQVqrTeSg1k8RS_TmmsN6viGYa4g640WtIdjfQqmpZ7lg_1YseLDfdVBWxB_kQjwH3lgX7rdHua6Hl62GayBQcbi7JdqkxdMI-J5DNbGE0FpK2rP1xZIXY/s1600/ld35.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg21ZEICOelps5ws9ys2wSALrQVqrTeSg1k8RS_TmmsN6viGYa4g640WtIdjfQqmpZ7lg_1YseLDfdVBWxB_kQjwH3lgX7rdHua6Hl62GayBQcbi7JdqkxdMI-J5DNbGE0FpK2rP1xZIXY/s640/ld35.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The first relay satellites arrive at Jool. I'm definitely guilty of setting up the WiFi before unpacking...</td></tr>
</tbody></table>
<div style="text-align: left;">
For the remainder of the ships, though, the Δv budget was tight enough that I definitely wanted to grab Tylo gravity assists on the way in. This created a bit of traffic as several ships would hit the Tylo gateway within days, or sometimes hours, of each other. To get captured using a gravity assist, I aimed to pass "in front of" Tylo, so that its gravity mostly pulls in a direction opposite my orbit and I feed it some of my kinetic energy. After some refinement, I also was able to target a captured orbit with a periapsis similar to the orbital radius of Laythe. From there, it's easy to get a low-energy intercept on the next orbit with just a couple small correction burns at periapsis and apoapsis.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY_gWcOy9LNLN1GMiKe7EYLR6ZCKihpm_GL0z_qGeoItvyV3KNaRIZa0QG9pPc-lkVD-xe_dp-JRqQ6JDVQ7F9wuVZBFtIm-P7GTMmbt3VsKRQMfAL7oC6F-yxvlDDqY1cERxLEJ3Hgdg/s1600/ld37.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="913" data-original-width="1600" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY_gWcOy9LNLN1GMiKe7EYLR6ZCKihpm_GL0z_qGeoItvyV3KNaRIZa0QG9pPc-lkVD-xe_dp-JRqQ6JDVQ7F9wuVZBFtIm-P7GTMmbt3VsKRQMfAL7oC6F-yxvlDDqY1cERxLEJ3Hgdg/s640/ld37.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Busy airspace (or, spacespace?) around the Tylo gateway.</td></tr>
</tbody></table>
<div style="text-align: left;">
Using gravity assist captures off Tylo, or in a few cases off Laythe itself, my average Δv from low Kerbin orbit to low Laythe orbit was about <span style="color: yellow;">3475m/s</span>, with a tolerance of about ±350m/s. This is quite a bit below the 4360m/s you get from the <a href="https://forum.kerbalspaceprogram.com/index.php?/topic/87463-13-community-delta-v-map-26/">subway map</a>, which would have been cutting it very close for some of my ships. As it is, all of the robotic fleet made it to low Kerbin orbit with fuel to spare and without having to do any aerobraking. Assuming all the Δv saved went into accelerating Tylo (and it wasn't on rails), its apoapsis would be raised by about 1nm.<br />
<br />
Getting to Laythe is not the same as landing on Laythe, though. It's a water world with only a few islands to target. I've <a href="https://scolton.blogspot.com/2014/03/ksp-mission-to-laythe-and-back.html">landed there before</a>, using a <a href="http://scolton.blogspot.com/2013/12/ksp-ascent-and-deorbit-simulators.html">custom deorbit burn tool</a> to target the island on the equator with the <a href="https://wiki.kerbalspaceprogram.com/wiki/File:Laythe_Topo_compressed.png">flattest terrain</a>. To hit that island, it makes sense to burn over the small island that's about 90º west of there. I set up each ship in a near-circular 100km equatorial orbit and then start a burn just as the ship passes over the coast of that island:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj95XxgNCrOfsdm30o1N6Y66SgUa-w9lYfbaEKrr4ZDWPK5NgZ5QTf60iZ_WgQHpXx6eAqNr2kU2EageOovkTtU8jXpN4Qa7Q0chrWG06gKM1L77PUiWjkZ7hB81cvyCR2dvwtXbbqiJ4U/s1600/ld44.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj95XxgNCrOfsdm30o1N6Y66SgUa-w9lYfbaEKrr4ZDWPK5NgZ5QTf60iZ_WgQHpXx6eAqNr2kU2EageOovkTtU8jXpN4Qa7Q0chrWG06gKM1L77PUiWjkZ7hB81cvyCR2dvwtXbbqiJ4U/s640/ld44.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Laythe deorbit burn over the small island on the equator, to hit the flat island about 90º to the east.</td></tr>
</tbody></table>
<div style="text-align: left;">
After the burn, the lander can ditch its propulsion module (which is mostly empty now and will burn up separately) and prep for entry. For the first phase of the landing, an inflatable heat shield protects the descent package from the initial atmospheric heating.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie9KaEcmxmphN_4Z7oFyX4y7Vr1Xmr9_UlyU8GdQoWNZMXTzOu_Qapz68bz3DXt5IpI_qyDzPFS6riTfm1RR5R23L6IRXBtqEidKa2h77ULPXg7Z3KGW61TS0VfYoBgLzE7P1mxjIs4nk/s1600/ld46.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie9KaEcmxmphN_4Z7oFyX4y7Vr1Xmr9_UlyU8GdQoWNZMXTzOu_Qapz68bz3DXt5IpI_qyDzPFS6riTfm1RR5R23L6IRXBtqEidKa2h77ULPXg7Z3KGW61TS0VfYoBgLzE7P1mxjIs4nk/s640/ld46.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Landing Phase 1: Using an inflatable heat shield to protect the payload while bleeding off some speed.</td></tr>
</tbody></table>
<div style="text-align: left;">
As the air gets thicker, the drag on the heat shield overcomes the ability of the reaction wheels to keep it facing forward, so the lander flips around. The fairing still provides thermal and aerodynamic protection for the payload, and the heat shield now becomes more of an air brake, bleeding off even more speed in preparation for the final descent.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN4_IgFn1bNstYGI8GNlEQon8C9YUcYNZAhyphenhyphenZTwOCEVrqDgLzMuMdne_ppQIimjkia8i0HD9DGb3AxD_qrQAhj5QExGPX475zNp9D600JinQ9zoYbfBLSid-Ctpj_DITurNaP9Rm1p0r0/s1600/ld47.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN4_IgFn1bNstYGI8GNlEQon8C9YUcYNZAhyphenhyphenZTwOCEVrqDgLzMuMdne_ppQIimjkia8i0HD9DGb3AxD_qrQAhj5QExGPX475zNp9D600JinQ9zoYbfBLSid-Ctpj_DITurNaP9Rm1p0r0/s640/ld47.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Landing Phase 2: The craft flips around, with the heat shield now acting as an air brake.</td></tr>
</tbody></table>
<div style="text-align: left;">
At about 3km AGL, the speed is low enough to jettison the fairing and deploy the main parachutes. The heat shield stays attached until the main chutes deploy, at which point it can be jettisoned in a controlled orientation so it doesn't crash back into the ship.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_jv4yFfzLA_CtwszAKmzHGP8MIJ42Ikperln0ENUUDWo_6CJdYUTLXrEBlTgKIbD60idScJTBvIjTYj-28EkutK5npXbIGczcB1POiMjI8IJN1DYwyQES5nKxdSsjGuZYkPlmK7ETFNQ/s1600/ld56.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_jv4yFfzLA_CtwszAKmzHGP8MIJ42Ikperln0ENUUDWo_6CJdYUTLXrEBlTgKIbD60idScJTBvIjTYj-28EkutK5npXbIGczcB1POiMjI8IJN1DYwyQES5nKxdSsjGuZYkPlmK7ETFNQ/s640/ld56.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Landing Phase 3: Fairing jettisoned, main chutes deployed, heat shield dropped.</td></tr>
</tbody></table>
<div style="text-align: left;">
Finally, at about 300m AGL, the descent engines kick in and bleed off the final bit of vertical velocity. They don't have much fuel, so the burn has to be timed pretty well. I use the AeroGUI's AGL indicator and the lander's shadow to judge it.</div>
</div>
<div style="text-align: left;">
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOgtS4KMV8JGy85lbezkWlOIKo9npq0mk0vJBrRPngNd9V0yElvsDH6wlqvFJDL7UfNDTn74oEqNQETiWEguAKGfFi-3-0hxRHnJbNGFzFYWJu3odJ8vf8VERyo7EOmP0k-ikUEXXeGvs/s1600/ld63.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOgtS4KMV8JGy85lbezkWlOIKo9npq0mk0vJBrRPngNd9V0yElvsDH6wlqvFJDL7UfNDTn74oEqNQETiWEguAKGfFi-3-0hxRHnJbNGFzFYWJu3odJ8vf8VERyo7EOmP0k-ikUEXXeGvs/s640/ld63.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Landing Phase 4: Powered descent. Kicks up a good amount of sand.</td></tr>
</tbody></table>
<div style="text-align: left;">
That's how things <i>should</i> go. But the first two landings were not quite perfect. I nearly overshot the landing zone on the first try, coming down less than 1km from the eastern shore. This is almost exactly where I landed my <a href="https://scolton.blogspot.com/2014/03/ksp-mission-to-laythe-and-back.html">first Laythe mission</a>, and I knew it was on a major slope. In the process of preparing for a potentially harrowing post-landing slide into the ocean, I forgot a few steps of the landing checklist and the descent engines didn't start up. The resulting ~15m/s impact was enough to break off the mining rig and fuel tank from the first LR1 rover down. But the drivetrain survived, so it could still act as a scout if it could get up the hill.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxQ9KxeAGPWji-UrMfO98KqttDP2XvYvnRSZ0GW1GzDsAP2PI8MZx7-MRX8fFCkqcmMYFfpZTk6GU0poTcvfFZU1Z7kwJTtfV5CLFib1eylQJrPFGh254IAhO_5cerj2BkR99sMX67KOY/s1600/ld41.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxQ9KxeAGPWji-UrMfO98KqttDP2XvYvnRSZ0GW1GzDsAP2PI8MZx7-MRX8fFCkqcmMYFfpZTk6GU0poTcvfFZU1Z7kwJTtfV5CLFib1eylQJrPFGh254IAhO_5cerj2BkR99sMX67KOY/s640/ld41.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The first (hard) landing on Laythe in this mission, dangerously close to the shore.</td></tr>
</tbody></table>
<div style="text-align: left;">
Having nearly overshot the landing zone into the ocean, I tweaked the deorbit burn a little (from 104m/s to 110m/s). However, this was a little too much tweaking and lander #2 wound up heading straight for the lake in the middle of this island. Luckily, this was a HAB1 lander, which has a little more fuel on board for the powered final descent. I managed to just barely hover-translate to the cliff edge overlooking the lake's eastern shore with no fuel to spare.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO4JWN1kPAGInozXCb9M2uFQNd28OSWXN94QLYFzGRsvktr5T7ilCNGpVGyA2YIU86ex7W1GN1LInGWJI-gc_4MXYHJ9S1JnjG7DI6CxOw_zc6hM7RhMVDvUHExmgkYp_l2lvlpVwjiMU/s1600/ld48.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO4JWN1kPAGInozXCb9M2uFQNd28OSWXN94QLYFzGRsvktr5T7ilCNGpVGyA2YIU86ex7W1GN1LInGWJI-gc_4MXYHJ9S1JnjG7DI6CxOw_zc6hM7RhMVDvUHExmgkYp_l2lvlpVwjiMU/s640/ld48.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Landing #2 involved some last-second piloting to steer away from the lake to the edge of a cliff.</td></tr>
</tbody></table>
<div style="text-align: left;">
Those two landings gave me the upper and lower limits for the deorbit burn. I used 108m/s as the burn for the remaining 12 landers, and they all touched down safely on the relatively flat land between the lake and the eastern shore.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3mC7UP0pDPcrac8Iz3plukG610mBv4dJH0KhSoB0Y5FUAj5DHvFqmkonBeIB60BSVftd7zw5zrMWhftNWbukcR2uoP_p6-QHDO0ow1HXVqhZihEuw3SVZZASMKtaNIUD-SWjrLVZd-RM/s1600/ld68.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3mC7UP0pDPcrac8Iz3plukG610mBv4dJH0KhSoB0Y5FUAj5DHvFqmkonBeIB60BSVftd7zw5zrMWhftNWbukcR2uoP_p6-QHDO0ow1HXVqhZihEuw3SVZZASMKtaNIUD-SWjrLVZd-RM/s640/ld68.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Typical landing zone after dialing in the exact deorbit burn.</td></tr>
</tbody></table>
<div style="text-align: left;">
I say relatively flat because it's still filled with sand dunes. They're no problem for the 6- and 8-wheeled rovers, but I need a 1-2km stretch of actually flat terrain to use as a space plane runway. I scouted for a while before settling on the strip marked out by the pink markers in the landing photo above. It's about 1.5km long and 300m wide, near the equator, and aligned well for west-to-east landings. It's completely flat in the crosswind direction and slightly sloped upward in the "upwind" landing direction. I'd prefer something flat in all directions, but this is the next best thing.<br />
<br />
By the end of the first wave, I could place the landers with about ±2km accuracy from orbit. But they are rovers, so it's easy to reposition them as needed. The LR1s all grouped together to form the corners of the runway, acting as visible markers for the space planes on approach. They're needed at the runway for refueling anyway, so this seems like the best place for them. In order to avoid excessive part counts in one location, I decided to move the HAB1s, the colony habitats, away from the runway and toward the lake. There, they could be assembled into housing groups.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzB8UkKMZW7LFjmK6TtT1Tj_QeDPpLbfr7txzUAEPI8zkm6W455eETNTOxXzYUcdeOr9Ixmu3JJFqkTi7q4LxjFgVgELyzN6kIKoAUdKA1n4nOgOaH69Ppz7T8y_wlNVmtItTCfYIyNc/s1600/ld72.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzB8UkKMZW7LFjmK6TtT1Tj_QeDPpLbfr7txzUAEPI8zkm6W455eETNTOxXzYUcdeOr9Ixmu3JJFqkTi7q4LxjFgVgELyzN6kIKoAUdKA1n4nOgOaH69Ppz7T8y_wlNVmtItTCfYIyNc/s640/ld72.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Setting up some modular housing on the dunes.</td></tr>
</tbody></table>
<div style="text-align: left;">
It's not a metropolis, but having a mobile and reconfigurable colony seems ideal on the sand dunes of an otherwise pretty desolate water planet. In total, 13.5 of the 14 rovers in the robotic fleet made it to the surface, and all 14 were able to find their way to each other and remotely set up the infrastructure for a colony. It'll be another year before the colony ships arrive in the second wave, but when they do, they'll have a place to stay - with a nice view.</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgymGtE5vG0W0Uvui6kThqT5mG0jZDrXEsoRhSNvXjp5Kzx3ILcYiS6x52JQlTar4vqN58E2-T46dcJT69U5z6iMPVARk3MLEwE5sIStOv8nbKqVOTwZS9jYhJz84xMDtngqD0VcN-dObk/s1600/ld62.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgymGtE5vG0W0Uvui6kThqT5mG0jZDrXEsoRhSNvXjp5Kzx3ILcYiS6x52JQlTar4vqN58E2-T46dcJT69U5z6iMPVARk3MLEwE5sIStOv8nbKqVOTwZS9jYhJz84xMDtngqD0VcN-dObk/s640/ld62.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-59593079522671149692019-09-28T17:10:00.004-04:002019-09-29T11:49:01.004-04:00Fast atan2() alternative for three-phase angle measurement.Normally, to get the phase angle of a set of (assumed balanced) three-phase signals, I'd do a <a href="https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_transformation"><span id="goog_581252223"></span>Clarke Transform<span id="goog_581252224"></span></a> followed by a <a href="https://en.wikipedia.org/wiki/Atan2">atan2</a>(β,α). This could be atan2f(), for single-precision floating-point in C, or some other <a href="http://www-labs.iro.umontreal.ca/~mignotte/IFT2425/Documents/EfficientApproximationArctgFunction.pdf">approximation</a> that trades off accuracy for speed. The crudest (and fastest) of these is a first-order approximation <span style="color: yellow; font-family: "times" , "times new roman" , serif; font-size: medium;">atan(x) ≈ (π/4)·x</span> <span style="text-align: center;"><span style="font-family: inherit;">which has maximum error of ±4.073º over the range {-1 ≤ x ≤ 1}</span>:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvw8qUwRAtPlaAZcH8JMjufmZ1d-pJPXQNuv2wG-aitOq3NWKhj2rsHnBRhATuArbg3EuPjh4D7LRZa0716QdWQF0EMdpXhANnhldUkGbNTb50BXgjloXi5LV-hl62wwjphsy2dS_VvDc/s1600/at00.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1455" data-original-width="1377" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvw8qUwRAtPlaAZcH8JMjufmZ1d-pJPXQNuv2wG-aitOq3NWKhj2rsHnBRhATuArbg3EuPjh4D7LRZa0716QdWQF0EMdpXhANnhldUkGbNTb50BXgjloXi5LV-hl62wwjphsy2dS_VvDc/s400/at00.png" width="377" /></a></div>
Interestingly, this isn't the best (<span style="color: #76a5af;">minimax</span> or <span style="color: #8e7cc3;">least mean square</span>) linear fit over that range. But it's pretty good and has zero error on both ends, so it can be stitched together into a continuous four-quadrant approximation that covers all finite inputs to the two-argument atan2(β,α):<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8Qvvfc7NXcKoElJBU5tuXGaQtNfK0QEj2qEYOpIYXJU1PRspMaRIZAXzM-oXTjyVee-rz3SpQDozIo1DPCuLhGJO2Ilwmo2lVf33dACTinpJn40QMXca-1woxbkxZ0ZVqymh-F1VMcww/s1600/at02.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1287" data-original-width="1504" height="341" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8Qvvfc7NXcKoElJBU5tuXGaQtNfK0QEj2qEYOpIYXJU1PRspMaRIZAXzM-oXTjyVee-rz3SpQDozIo1DPCuLhGJO2Ilwmo2lVf33dACTinpJn40QMXca-1woxbkxZ0ZVqymh-F1VMcww/s400/at02.png" width="400" /></a></div>
One common implementation determines the quadrant based on α and β and then runs the linear approximation on either <span style="font-family: inherit;">x = <span style="color: #ea9999;">β/α</span> or x = <span style="color: #9fc5e8;">α/β</span></span>, whichever is in the range {-1 ≤ x ≤ 1} in that quadrant. The combination of a quadrant offset and the local linear approximation determines the final result.<br />
<br />
It's possible to extend this method to three inputs, a set of three-phase signals assumed to be balanced. Instead of quadrants, the input domain is split based on the six possible sorted orders of the three-phase signals. Within each sextant, the middle input (the one crossing zero) is divided by the difference of the other two to form a normalized input, analogous to selecting x = β/α or x = α/β in the atan2() implementation:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwENr2oXMNxm8YH40P7DZhpO_vuHfmWm9Z-8kNdPJSzR4u4D9AFzJMEd3q268Ol8ITjhhejeHF8OYn7pCNxbuYvdUsk36bvXbtEtAvouMep8KKqoVqE_a3cL-skYMpfOQcSidRyCer204/s1600/at03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="788" data-original-width="1600" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwENr2oXMNxm8YH40P7DZhpO_vuHfmWm9Z-8kNdPJSzR4u4D9AFzJMEd3q268Ol8ITjhhejeHF8OYn7pCNxbuYvdUsk36bvXbtEtAvouMep8KKqoVqE_a3cL-skYMpfOQcSidRyCer204/s400/at03.png" width="400" /></a></div>
This normalized input, which happens to range from -1/3 to 1/3, is multiplied by a linear fit constant to create the local approximation. To follow the pattern of the four-quadrant approximation, a constant of <span style="font-family: "times" , "times new roman" , serif; font-size: medium;"><span style="color: cyan;">π/2</span></span><span style="color: yellow; font-family: "times" , "times new roman" , serif; font-size: large;"> </span>gives a fit that's not (minimax or least mean square) optimal, but stitches together continuously at sextant boundaries. As with the atan2() implementation, the combination of a sextant offset and the local approximation determine the final result.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzhwuEOI4Y9nrqC6a6uaCdJ0ocH8ldrAkok-nRpErxvSPfaSBXz4dtnvQjapRwhCqNXZ6dbCJFB4fJn5-rK2OQD3p2_SmrNmK-0pKqrtd3bPavKFMRn0xodcpG-ntzq0RNq7AjQmoRzGY/s1600/at04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="802" data-original-width="1600" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzhwuEOI4Y9nrqC6a6uaCdJ0ocH8ldrAkok-nRpErxvSPfaSBXz4dtnvQjapRwhCqNXZ6dbCJFB4fJn5-rK2OQD3p2_SmrNmK-0pKqrtd3bPavKFMRn0xodcpG-ntzq0RNq7AjQmoRzGY/s400/at04.png" width="400" /></a></div>
For this three-phase approximation the maximum error is ±1.117º, significantly lower than the four-quadrant approximation. If starting from three-phase signals anyway, this method may also be faster, or at least nearly the same speed. The conditional section for selecting a sextant is more complex, but there are fewer intermediate math operations. (Both still have the single pesky floating-point divide for normalization.)<br />
<br />
To put this to the test, I tried directly computing the phase of the three <a href="https://scolton-www.s3.amazonaws.com/motordrive/sensorless_gen1_Rev1.pdf">flux observer</a> signals on <a href="https://scolton.blogspot.com/search/label/TinyCross">TinyCross</a>'s dual motor drive. This usually isn't the best way to derive sensorless rotor angle: An angle tracking observer or PLL-type method can do a better job at filtering out noise by enforcing physical bandwidth constraints. But for this test, I just compute the angle directly using either atan2f(β,α) or one of the two approximations above.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigJqcxMinj6mdQRLP3sl7CLlvzqjOZJQ6bBcpej2on2vw49JofkijKuFiyDMsm-V-Uis5AUaBoU-qrd6yMP77A-UdGXWM1LJ4mWy9KSCZRNE_83pVXnrf8nsQH20c9h5UXWw-xKTz47R4/s1600/at06.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="137" data-original-width="706" height="77" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigJqcxMinj6mdQRLP3sl7CLlvzqjOZJQ6bBcpej2on2vw49JofkijKuFiyDMsm-V-Uis5AUaBoU-qrd6yMP77A-UdGXWM1LJ4mWy9KSCZRNE_83pVXnrf8nsQH20c9h5UXWw-xKTz47R4/s400/at06.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Computation times for different angle-deriving algorithms.</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
The three-phase approximation does turn out to be a little faster in this case. To keep the comparison fair, I tried to use the same structure for both approximations: the quadrant/sextant selection conditional runs first, setting bits in a 2- or 3-bit code. That code is then used to look up the offset and the numerator/denominator for the the local linear approximation. This is running on an STM32F303 at 72MHz. The PWM loop period is 42.67μs, so a 1.5-2.0μs calculation per motor isn't too bad, but every cycle counts. It's also "free" accuracy improvement:</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid8xxBiqeaw7Yrp3zkiE00thJ27ef5PaEto6u1RA51X3zxJ6pQFhMnAw9ROwL_H4kaHDXfG1k6kXrO86q4AdKMHAI5YJzXxpcuLK7NYT6WiN8XVn8baGM5vhELUwwpUUDtgsV_iUtF9Po/s1600/at05.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="990" data-original-width="1600" height="396" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid8xxBiqeaw7Yrp3zkiE00thJ27ef5PaEto6u1RA51X3zxJ6pQFhMnAw9ROwL_H4kaHDXfG1k6kXrO86q4AdKMHAI5YJzXxpcuLK7NYT6WiN8XVn8baGM5vhELUwwpUUDtgsV_iUtF9Po/s640/at05.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: left;">
The ±4º error ripple in the four-quadrant approximation shows up clearly in real data. The smaller error in the three-phase approximation is mostly lost in other noise. When the error is taken with respect to a post-computed atan2f(), the four-quadrant approximation looks less noisy. But I think this is just a mathematical symptom. When considering error with respect to an independent angle measurement (from Hall sensor interpolation), they show similar amounts of noise.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I don't have an immediate use for this, since TinyCross is primarily sensored and the flux signals are already <a href="http://scolton.blogspot.com/2019/09/tinycross-first-test-drive-and.html">synchronously logged</a> (for diagnostics only). But clock cycle hunting is a fun hobby.</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com5tag:blogger.com,1999:blog-8200098102909041178.post-90971080505662875162019-09-24T01:34:00.000-04:002019-09-24T01:38:04.940-04:00TinyCross: First Test Drive and Synchronous Data Logging<div class="separator" style="clear: both; text-align: left;">
With the front wheel drive complete and the steering wheel control board working, it's finally time for a first test drive:</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/LhocJXDgiWc" width="640"></iframe>
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I've been waiting over a year to see if this mountain bike air shock suspension setup would work, and it looks like it does! I haven't done any tuning on it besides setting the preload, but it handles my pretty beat up parking lot nicely, absorbing bumps that would have broken <a href="http://scolton.blogspot.com/p/cap-kart.html#tinykart">tinyKart</a> in minutes. The steering linkage also seems okay, with good travel and minimal bump steer. There are still some minor mechanical improvements I want to make, but it's nice to see the suspension concept in action after all this time.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I started with front wheel drive so I could see if the motor drive had any Flame Emitting Transistors, but happily it did not. It's the same <a href="https://scolton.blogspot.com/2018/09/tinycross-electron-control-unit.html">gate drive design</a> that I use on everything and it always just works, so I shouldn't be surprised anymore. But I am asking a lot of the <a href="https://www.onsemi.com/products/discretes-drivers/mosfets/fdmt80080dc">FDMT80080DC</a> FETs (just one per leg), so I'm working my way up to 120A (peak, line-to-neutral) phase current incrementally. The above test is at 80A and the FETs seem happy, although the motors do get pretty warm already. They might need some i²t thermal protection to handle 120A peaks.</div>
<h4 style="clear: both; text-align: left;">
Synchronous Data Logging</h4>
<div>
One of the early lessons I learned in building motor drives is to always log data. Nothing ever works perfectly on the first try, but having data logging built in from the start is the best way I know of to quickly diagnose problems. A lot of the stuff that happens in a motor drive is faster than typical data logging can capture, but a lot of it is also periodic. By synchronizing the data collected to the rotor electrical angle, its possible to reveal detailed periodic signals even with relatively low frequency (50Hz) logging to an SD card. As a quick example, here's a standard data logger plot of motor phase currents over time:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixsxfaw_SDVR2xjCn_-aJO3m0tR-P74d-mY8kgu4JxJsYLSjYsg-AIgdfoFpNFlVF5kt8IPTB4fvZUwO8XWpe5FOMygd9IyJLh4FslU40_XnXKsuF1wlCET2uG4ga9hdMoz7o62jmrPvc/s1600/19-09-15_03.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="989" height="412" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixsxfaw_SDVR2xjCn_-aJO3m0tR-P74d-mY8kgu4JxJsYLSjYsg-AIgdfoFpNFlVF5kt8IPTB4fvZUwO8XWpe5FOMygd9IyJLh4FslU40_XnXKsuF1wlCET2uG4ga9hdMoz7o62jmrPvc/s640/19-09-15_03.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Phase current vs. time, pretty boring.</td></tr>
</tbody></table>
<div style="text-align: left;">
This type of plot shows the drive cycle, with periods of high current during acceleration (or braking) and periods of near zero current when coasting or stopped. And it shows that phase currents sometimes exceed 100A even with an 80A command. But a plot of Q-axis (torque-producing) current, which is already synchronous, could give a better summary of this information. The time resolution (40ms) isn't fine enough to show the AC signals.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
However, each set of three phase currents is also stamped with a rotor electrical angle measured at the same time (within about 10 microseconds). Cross-plotting the phase currents against their angle stamp, instead of against time, reveals a much more interesting view of the data:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZMB65a0k1L5YGUURaIMXe53z9EyzJVokj6xGHlqSRXx2Wv_JjNpzeP5TBplPW2jPqyVq5m_w5jX88nhGEGV4_iA5GEH6lREr7PqGBJfpC6h1QFKW5KURatQxFoifVIb6DttF3AskSiJU/s1600/19-09-15_04.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="640" data-original-width="989" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZMB65a0k1L5YGUURaIMXe53z9EyzJVokj6xGHlqSRXx2Wv_JjNpzeP5TBplPW2jPqyVq5m_w5jX88nhGEGV4_iA5GEH6lREr7PqGBJfpC6h1QFKW5KURatQxFoifVIb6DttF3AskSiJU/s640/19-09-15_04.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Same data, different meaning.</td></tr>
</tbody></table>
<div style="text-align: left;">
Now it's possible to see the three phase current waveforms separated by 120edeg. The peaks are at 0º (Phase A), 120º (Phase B), and -120º (Phase C). There are also negative peaks at the same angles, where braking is occurring. Most interestingly, the shape of the current waveforms at 80A peak is revealed to be asymmetric and far from sinusoidal.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The angular resolution of this type of waveform capture is only limited by the angle measurement, regardless of logging frequency. By contrast, the fastest it would be possible to log a continuous waveform would be at the PWM frequency (23.4kHz, in this case), which gives an speed-dependent angular resolution of 11.3edeg per 1000rpm. It would become difficult to resolve the shape of the current waveform at high speeds. There's always a trade-off, though: Synchronizing low-speed log data with angle stamps is only able to show the average shape of long-term periodic signals. It would not catch a glitch in a single cycle of the phase currents.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
While the phase current shape is interesting, the position of the peaks is just a consequence of the current controller. Zero electrical degrees is defined (by me, arbitrarily) as the angle at which Phase A's back EMF is at a peak. The current controller <a href="http://scolton.blogspot.com/2009/11/everything-you-ever-wanted-to-know.html">aligns the Phase A current with the Phase A back EMF</a> for maximum torque per amp. So the phase current plot shows that the current controller is doing its job. This information is also captured by the already-synchronous Q-axis and D-axis current signals:</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQDYxQFBZ9e7pwu4marUZWAof4RxOUWNSfCEDdcxWJrPAFGekBYBm7Kf9rwit61dkBRiyQsireaiahMlWEa61WRTLw59heyUdre71ZyK_zkSHHN_enJHEQAzpesXox5v4Qb8qv9KF5NnU/s1600/19-09-15_05.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="973" data-original-width="1600" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQDYxQFBZ9e7pwu4marUZWAof4RxOUWNSfCEDdcxWJrPAFGekBYBm7Kf9rwit61dkBRiyQsireaiahMlWEa61WRTLw59heyUdre71ZyK_zkSHHN_enJHEQAzpesXox5v4Qb8qv9KF5NnU/s640/19-09-15_05.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Q-axis and D-axis current plotted against time.</td></tr>
</tbody></table>
<div style="text-align: left;">
The Q-axis current represents torque-producing current, aligned with the back EMF, and is the current being commanded by the throttle input. The D-axis current is field-augmenting (or weakening, if negative) current and doesn't contribute to torque production. In this case, the current controller seeks to hit the desired Q-axis current and keep the D-axis current at zero. It does this by varying the voltage vector applied to the motor. More on this later. The Q-axis and D-axis currents are rotor-synchronous values, so they already convey the magnitude and phase of the phase currents, just not the actual shape.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
All of this is based on the assumption that the measured rotor angle is correct, i.e. properly defined with respect to the permanent magnets. On this kart, I'm using magnetic rotary sensors mounted to the motor shafts that communicate the rotor angle to the motor controller via SPI and <a href="http://scolton.blogspot.com/2018/09/tinycross-electron-control-unit.html">optically-isolated emulated Hall sensor signals</a>. But it's also possible to measure the rotor angle with a <a href="http://scolton.blogspot.com/search/label/sensorless">flux observer</a>, as long as the motor is spinning sufficiently fast. I have this running in the background, logging flux estimates for each phase.<br />
<br />
Again, plotting flux against time doesn't give a whole lot of information. It's interesting to see the observer converge as speed increases from zero at the start, and the average amplitude of about 5mWb is consistent with the motor's rpm/V constant and measured back-EMF. But the real value of this data comes from cross-plotting against the sensor-derived rotor angle:</div>
<div style="text-align: left;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Jsb_4YxJtafSB_i-7hNrGgaqj8zyfyF7h-yECBadBkJuEDuUhFiZCVujQk2f-7m_06ycEHDJbLc2hey7nsH03i2OzQLl0WJjv1UHYcPDcNM1I-gmkntlaU0L-aN_aSoyl1YEhn3YnVM/s1600/19-09-15_07.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="668" data-original-width="1002" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Jsb_4YxJtafSB_i-7hNrGgaqj8zyfyF7h-yECBadBkJuEDuUhFiZCVujQk2f-7m_06ycEHDJbLc2hey7nsH03i2OzQLl0WJjv1UHYcPDcNM1I-gmkntlaU0L-aN_aSoyl1YEhn3YnVM/s640/19-09-15_07.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Cross-plotting against sensor-derived electrical angle shows substantial offset between the two motors.</td></tr>
</tbody></table>
</div>
<div style="text-align: left;">
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
<div style="text-align: left;">
The flux from Phase A should cross zero when its back EMF is at its peak, i.e. at an electrical angle of 0º in my arbitrarily-defined system. So, the front-right motor is more correct. The front-left is offset by about 30-45edeg, which is enough to start causing significant differences in torque. Indeed I had noticed some torque steer during the first test drives, which is what prompted me to do the sensor/sensorless angle comparison in the first place.</div>
<div style="text-align: left;">
<br />
Since I have all three phases of flux, I can estimate the flux vector angle with some math and compare it to the sensor-derived rotor angle:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilMWNbv2VKWFrRm_Fv6vNP6l3CUEpqNTyttqyV5PGIiZUkB8zjGxfW2vSkwgSlUryvg9HXIjgK54BIyPtg4HZ8snLR7_2sA9dDAlAZ0bJiClrr7VATWPbkGT16fErAMhqPQ29ZfHi2pYw/s1600/19-09-15_08.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1267" data-original-width="1600" height="506" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilMWNbv2VKWFrRm_Fv6vNP6l3CUEpqNTyttqyV5PGIiZUkB8zjGxfW2vSkwgSlUryvg9HXIjgK54BIyPtg4HZ8snLR7_2sA9dDAlAZ0bJiClrr7VATWPbkGT16fErAMhqPQ29ZfHi2pYw/s640/19-09-15_08.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Digging into flux angle offset of the front-left motor a little more.</td></tr>
</tbody></table>
Both motors have some variation in flux angle offset, but the front-left varies more and is further from the nominal 90º. Except...when it's not. There are two five-second intervals where the average offset of the front-left flux looks like it returns to nearly 90º, both occurring either during or just after applying negative current. However, there's one more negative current pulse, earlier in the test drive, that does not have a flux angle shift. My troubleshooting neural network has been trained over many project iterations to interpret this as the signature of a mechanical problem.<br />
<br />
Sure enough, I was able to grab the rotor of the front-left motor and twist with hand strength only (< 5Nm) enough to make the shaft move relative to the rotor can. It only moved about 5º, but that's 35edeg, which is about the offset I had been seeing in the data. The press fit had failed and it was relying on back-up set screws on flats to keep from completely slipping. I suspect this won't be the last motor to fail in this way. I pressed out the shaft, roughed up the surface a little, and pressed it back in with some Loctite 609. I also drilled a hole in the back that can potentially be tapped as a back-up plan. And finally I recalibrated everything and marked the shaft so I'll know if it slips again.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkL9RaJJVv1iEASxGXyQ2DKQ5tAjyr4UhKjtQK_Is3sJWa02GJ-YDHCV97y61Hy0s9uMAfHu7LJCoQz-BNHHtoCOy2dybYt26TRw-E3T5OFoui0Im0M6ixqSnxJAyjZSFoXcOwAG0ra18/s1600/tc74.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkL9RaJJVv1iEASxGXyQ2DKQ5tAjyr4UhKjtQK_Is3sJWa02GJ-YDHCV97y61Hy0s9uMAfHu7LJCoQz-BNHHtoCOy2dybYt26TRw-E3T5OFoui0Im0M6ixqSnxJAyjZSFoXcOwAG0ra18/s640/tc74.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Reworked shaft, with a 1/4-20 tap drill (not going to tap it unless I absolutely have to), roughed surface, and press-fit augmented with Loctite 609, which should be good up to 25Nm for this surface area (4-5x margin).</td></tr>
</tbody></table>
<div style="text-align: left;">
After a few more test drives, it looks like it's holding. The front-left flux vs. sensor-derived angle looks much closer to the correct phase as well:</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVYOEUzUaEgFoi7LH0IjS5Sk54c0DGeHfKM6nprrQXp1C97rN_NpFFJpY5h58Tz4Ss84qDcQlo2wyDRHXlWQVOSjilfUEW5ALjblxCXoqpq_YXPRvdchN3ax6ac0UXskNPxbj0n1vFTAM/s1600/19-09-22_00.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1110" data-original-width="1600" height="444" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVYOEUzUaEgFoi7LH0IjS5Sk54c0DGeHfKM6nprrQXp1C97rN_NpFFJpY5h58Tz4Ss84qDcQlo2wyDRHXlWQVOSjilfUEW5ALjblxCXoqpq_YXPRvdchN3ax6ac0UXskNPxbj0n1vFTAM/s640/19-09-22_00.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Phase A flux vs. sensor-derived angle after shaft rework.</td></tr>
</tbody></table>
<div style="text-align: left;">
There's still a +/-10edeg offset from nominal, which could be from calibration accuracy or static biases like normal shaft twisting. It might be worth investigating more, but it's not enough offset to create any noticeable torque steer on the front wheel drive, so I'm satisfied for now. I will preemptively do the same rework on the remaining three motor shafts.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
One other interesting cross-plot to look at is the Q- and D-axis voltage as a function of speed. I mentioned above that the current controller attempts to align the current vector with the back EMF vector by manipulating the voltage vector, the basis of field-oriented control. Due to the electrical time constant (L/R) of the motor, the voltage must lead the back EMF by a varying amount. This shows up as negative D-axis voltage increasing in magnitude with speed (and current).</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieyd05zY2tRt3PfER4qJxwqXySs7pi0LA1mUQL5zq_tBitikMfd2CvyYgSZRXYobtWdJ8fq8-x5yLY2GgFwhV2MI7oU0bdpu0voHb6tvZS8nt3eGl7La7CeHSsDsWPOwDgNmp9Xuj3Pk8/s1600/19-09-22_01.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="628" data-original-width="901" height="446" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieyd05zY2tRt3PfER4qJxwqXySs7pi0LA1mUQL5zq_tBitikMfd2CvyYgSZRXYobtWdJ8fq8-x5yLY2GgFwhV2MI7oU0bdpu0voHb6tvZS8nt3eGl7La7CeHSsDsWPOwDgNmp9Xuj3Pk8/s640/19-09-22_01.gif" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Jitter 3D plot of the voltage vector operating curve.</td></tr>
</tbody></table>
<div style="text-align: left;">
At 80A and 2500erad/s (~3400rpm and ~27mph), the voltage vector is already leading by 45º, with 12V on both axes. This gives me a rough estimate for the motor's synchronous inductance.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOzyfo1nbhQXFL2I1ww8tZ2R_Hmzx8QlAeDBph_OV0A6hyyN88vXn3eT3SzhEzlpsucHBFU4u1wNfsjOSkGOCXCTPU6JPlI4POh0Zbnx-US0kskJwVPVDxk1-kigY15ld0YrUa-oySc_o/s1600/19-09-22_01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="193" data-original-width="514" height="120" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOzyfo1nbhQXFL2I1ww8tZ2R_Hmzx8QlAeDBph_OV0A6hyyN88vXn3eT3SzhEzlpsucHBFU4u1wNfsjOSkGOCXCTPU6JPlI4POh0Zbnx-US0kskJwVPVDxk1-kigY15ld0YrUa-oySc_o/s320/19-09-22_01.png" width="320" /></a></div>
<div style="text-align: left;">
Along with the measured resistance (32mΩ) and flux amplitude (5mWb), this is all that's required for a first-order motor model, and thus a torque-speed curve. Running this through the gear ratio, the force-speed curve at the ground should look something like:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3yD9wSn6oQjTUOJtNVwEjcZp83CIrl2onR1-xWDDw5HxbkWlW7rGR8J6AkLefAPh-BdOxi8aYAEl6rry2KcS49mL2RrRyMVY-jFV2EBEVi2HC-ZppP6z6amAI8jtarxp33AALz3kZu-Y/s1600/19-09-22_03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="718" data-original-width="1526" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3yD9wSn6oQjTUOJtNVwEjcZp83CIrl2onR1-xWDDw5HxbkWlW7rGR8J6AkLefAPh-BdOxi8aYAEl6rry2KcS49mL2RrRyMVY-jFV2EBEVi2HC-ZppP6z6amAI8jtarxp33AALz3kZu-Y/s640/19-09-22_03.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
The inductance has a large impact on the maximum speed at which 120A can be driven into the motor in-phase with the back EMF. This determines the maximum power, since above this speed the force drops off faster than the speed increases. The top speed is wherever on the curve the drag forces equal the motor force, probably in the 40-45mph range. This is all without using third harmonic injection, which gives an extra 15% voltage overhead (for the cost of higher peak battery power, of course). If I do turn that on, it will probably come with a gear ratio change to put that extra 15% toward more torque, not more speed.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
That's all I wanted to check before building up the second motor controller for the rear wheel drive. I'm very eager to see how it handles with 4WD, and how close to this force-speed curve I can actually get.</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com2tag:blogger.com,1999:blog-8200098102909041178.post-7612329123232298322019-09-04T22:57:00.000-04:002019-09-05T11:59:59.038-04:00CMV12000 Full-Speed (38.4Gb/s) Read-In on Zynq Ultrascale+In my original <a href="http://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">Freight-Train-of-Pixels</a> post, I explored three main challenges of building a 3.8Gpx/s imager: the source, the pipe, and the sink. Working backwards, the sink is an NVMe SSD that (<a href="http://scolton.blogspot.com/2019/07/benchmarking-nvme-through-zynq.html">hopefully</a>) will be capable of 1GB/s writes. The pipe is a ~5:1 wavelet compression engine that has to make 3.8Gpx/s = 1GB/s in realtime, with minimal effect on image quality. And the source is the CMV12000 image sensor that relentlessly feeds pixel data into this machine. This post focuses on the source, and specifically the read-in mechanism implemented on a Zynq Ultrascale+ SoC for the 38.4Gb/s of LVDS data from the sensor.<br />
<h4>
Physical Interface</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz0zwfs81tIQ_BDScivkxC7-6gKKkSQjB3vKaglN6JqnAObD6-XptWiG-85OcBtQKeGWkvXeiEtymVce5064Pp6Xo6UHGlhnIyUA4HrPNK_FdU1LRu8xJNhKJGXONDFrOlh_qrrGNSpik/s1600/c04.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1098" data-original-width="1600" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz0zwfs81tIQ_BDScivkxC7-6gKKkSQjB3vKaglN6JqnAObD6-XptWiG-85OcBtQKeGWkvXeiEtymVce5064Pp6Xo6UHGlhnIyUA4HrPNK_FdU1LRu8xJNhKJGXONDFrOlh_qrrGNSpik/s400/c04.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Source: Breaking out the CMV12000's 64 LVDS pairs was interesting.</td></tr>
</tbody></table>
<div style="text-align: left;">
The pixel data interface on the CMV12000 is 64 LVDS pairs, each operating at (up to) 300MHz DDR (600Mb/s). In the context of FPGA I/O interfaces, 300MHz DDR is really not that fast. It's just a lot of inputs. Most Zynq Ultrascale+ SoCs have enough LVDS-capable package pins to do this, but it took some searching to find a carrier board that breaks out enough of them to headers. I'm using the <a href="https://shop.trenz-electronic.de/en/Products/Trenz-Electronic/TE08XX-Zynq-UltraScale/TE0803-Zynq-UltraScale/">Trenz Electronic TE0803</a>, specifically the ZU4CG vesion, which breaks out a total of 72 HP LVDS pairs from the ZU+.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The physical interface for LVDS is a 100Ω differential pair. At 300MHz DDR, the length-matching requirements are not difficult. A bit is about 250mm long, so millimeter-scale mismatches due to an uneven number of 45º left and right turns are not a big deal; no meandering is really needed for intrapair matching. Likewise, I felt it was okay to break some routing rules by splitting a pair for a few millimeters to reach inner CMV12000 pins, rather than pushing trace/space limits to squeeze two traces between pins.<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg39HKLxniHQwqobJERmxEnJ_hbF_tEwGUqTZTeDmPn1VXbx6cXOykYPfvyApixuu2Pc5noVRxVXHxVVS-TTh9KfHejpQZQjSUVRFcUqtpgNgvnjMk5Ag_J6UXKK0De5wkSrn8BYd7wt0E/s1600/c65.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1259" data-original-width="1600" height="502" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg39HKLxniHQwqobJERmxEnJ_hbF_tEwGUqTZTeDmPn1VXbx6cXOykYPfvyApixuu2Pc5noVRxVXHxVVS-TTh9KfHejpQZQjSUVRFcUqtpgNgvnjMk5Ag_J6UXKK0De5wkSrn8BYd7wt0E/s640/c65.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Routing of the LVDS pairs to TE0803 headers. Pairs are length-matched to within ~1mm, but no interpair matching was attempted. The FPGA must deal with interpair length differences as well as the CMV12000's large internal skew.</td></tr>
</tbody></table>
<div style="text-align: left;">
Interpair skew <i>is</i> still an issue. For ease of routing, no interpair length matching was attempted, resulting in length differences of as much as 30% of a bit interval. But this isn't even the bad news. The CMV12000 has a ~150ps skew between <i>each</i> channel from 1-32 and 33-64. That means that channels 32 and 64 are ~4.7ns behind channels 1 and 33, a whopping 280% of a bit interval. It would be silly to try to compensate for this with length matching, since that's equivalent to about 700mm at c/2!</div>
<h4 style="text-align: left;">
Deserialization and Link Training</h4>
<div>
For a brief moment after reading about the CMV12000's massive interchannel skew, I thought I might be screwed. FPGA inputs deal with skew by using adjustable delay elements to add delay to the edges that arrive early, slowing them all down to align with the most fashionably late edge. But the delay elements in the Zynq Ultrascale+ are only guaranteed to provide up to 1.1ns of delay. It's possible to cascade the unused output delay elements with their associated input delay elements, but that's still only 2.2ns.<br />
<br />
But I don't need to account for the whole 4.7ns of interchannel skew; I only need to reach the same phase angle in the next bit. At 600Mb/s, that's only 1.67ns away. Delays larger than this can be done with bit slipping, as shown below. Since this still relies on the cascaded delay elements to span one full bit interval, an interesting consequence is that a <i>minimum</i> speed is imposed (about 450Mb/s for 2.2ns of available delay). So I guess it's go fast or go home...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia-nzY-LowYXjbqHLKGcqFoK3jUGI8KmlZwObzx5uxT_Ny73G5mRE2r1frIFBn_9Q0JNyfgHBO0Jg1KU-k2MzkLoStqALV8hS-e2i3zgldUwtBGoqOeBYtfFQh62mbtmRvpy2JQSjmrnM/s1600/c66.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="328" data-original-width="1600" height="129" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia-nzY-LowYXjbqHLKGcqFoK3jUGI8KmlZwObzx5uxT_Ny73G5mRE2r1frIFBn_9Q0JNyfgHBO0Jg1KU-k2MzkLoStqALV8hS-e2i3zgldUwtBGoqOeBYtfFQh62mbtmRvpy2JQSjmrnM/s640/c66.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Channels are aligned using an adjustable delay of up to one bit period and integer bit slipping in the deserialized data.</td></tr>
</tbody></table>
The Ultrascale+ deserializer hardware supports up to 1:8 (byte) deserialization from DDR inputs. The bit slip logic selects a new byte from any position in the 16-bit concatenation of the current and previous raw deserialized bytes. The combination of the delay value and integer bit slip offset independently align each channel.</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div style="text-align: left;">
A complication is that the CMV12000 has 8-, 10-, and 12-bit pixel modes, with the highest readout efficiency in the default 10-bit mode. To go from 8-bit deserialized data to 10-bit pixel data requires building a "gearbox", a nomenclature I really like. An 8:10 gearbox can be built pretty easily with just a few registers:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfMdjLxqDVQgOmeav2kvXX3fOBmIVIFBsXybNrRp_mRUSZOndH4SjJ25F8a9f-a8Aox0VSXdjh34kN2oa8Jn8qE1zbT2VIFqUpK_EFlHfaz_eNsK_oJMXvb8iROcY0s3ky6jFBfhSWe1o/s1600/c67.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="468" data-original-width="1555" height="192" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfMdjLxqDVQgOmeav2kvXX3fOBmIVIFBsXybNrRp_mRUSZOndH4SjJ25F8a9f-a8Aox0VSXdjh34kN2oa8Jn8qE1zbT2VIFqUpK_EFlHfaz_eNsK_oJMXvb8iROcY0s3ky6jFBfhSWe1o/s640/c67.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">An 8:10 gearbox, with four states corresponding to alignment of the 10-bit output within two adjacent 8-bit inputs.</td></tr>
</tbody></table>
<div style="text-align: left;">
The gearbox cycles through four states, registering a 10-bit output from an offset of {0, 2, 4, or 6} within two adjacent 8-bit inputs to pick out whole pixels from the data. This looks simple enough, but there's a subtlety in the fact that five bytes must cycle through the registers for every four pixels. In other words, the input clock (byte_clk) is running 5/4 as fast as the output clock (px_clk). The two clocks must be divided down from the same source (the LVDS clock in this case) to ensure that timing constraints can be evaluated. Additionally, to work as pictured above, the phase of the two clocks must be such the "extra" byte shift occur between states 3 and 0.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The overall input module is pretty tiny, which is good because I have to instantiate 65 of them (64 pixel channels and one control channel). They're built into an AXI-Lite slave peripheral with all the per-channel tweakable parameters as well as the final 10-bit pixel outputs mapped for the ARM to play with. The CMV12000 outputs training data on the pixel channels any time they're not being used to send real data. So, my link training process is:</div>
<div style="text-align: left;">
</div>
<ol>
<li>Find the correct phase for the px_clk so that, as described above, the gearbox works properly. Incorrect phase will result in flickering pixel data as the byte shifts occur in the wrong place relative to the gearbox px_clk state machine. I'm not sure why this phase changes from reset to reset. It's the same value for all 65 channels, so I feel like there should be a way to have it start up deterministically. But for now it's easy enough to try all four values and see which one produces constant data.<br /> </li>
<li>On each channel, set the sampling point by sweeping through the adjustable delay values looking for an eye center. (Or, since it's not guaranteed that a complete eye will be contained in the available delay range, a sampling point sufficiently far from eye edges.)<br /> </li>
<li>On the control channel, set the bit slip offset to the value between 3 and 12 that produces the expected training value. This covers all ten possibilities for phasing of the pixel data relative to the deserializer. Note that this requires registering and concatenating three deserialized bytes, rather than two as pictured in the bit slip example above.<br /> </li>
<li>On each pixel channel, set the bit slip offset to the value closest to the control channel bit slip offset that produces the expected training value. It should be within ±3 of the control channel bit slip offset, since that's the maximum interchannel skew.</li>
</ol>
<div>
This only takes a fraction of a second, so it can easily be done on start-up or even in between captures to protect against temperature-dependent skew. By looking at the total delay represented by the delay tap values and bit slip offsets, it's clear that the CMV12000's interchannel skew is the dominant factor and that the trained delays roughly match the datasheet skew specification of 150ps per channel:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHfI5gzqJGPhFg46SmWR7u-T38k_I-qmvccsZzwXwk8khSi98WIwGbUxoBuBJW3J9oS-x1WzqKLUFwXkV-mpQQGpk6MKsX98mZ-Tgrj7SGQKU-VLaJx7qa200ACFVVkY9_CAksEhHrEes/s1600/c37.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="663" data-original-width="1001" height="263" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHfI5gzqJGPhFg46SmWR7u-T38k_I-qmvccsZzwXwk8khSi98WIwGbUxoBuBJW3J9oS-x1WzqKLUFwXkV-mpQQGpk6MKsX98mZ-Tgrj7SGQKU-VLaJx7qa200ACFVVkY9_CAksEhHrEes/s400/c37.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Total CMV12000 channel delays measured by training results.</td></tr>
</tbody></table>
<div style="text-align: left;">
That's the hard part of the source done, with less trouble than I expected. The output is a 60MHz px_clk and 65 10-bit values that update on that clock. This will be the interface to the middle of the pipeline, the wavelet engine. But I need to be able to test the sensor before that's complete, and more than 64 pixels at a time. Without the compression stage, though, that means writing data at full rate to external DDR4 attached to the ZU+. Although it's a throwaway test, I will need to write to that RAM (at a lower rate) after the compression stage anyway, so this would be good practice.</div>
<h4 style="text-align: left;">
RAMMING SPEED</h4>
<div>
The ZU4CG version of the TE0803 has 2GB of 2400MT/s DDR4 configured as x64. That's over 150Gb/s of theoretical memory bandwidth, so the 38.4Gb/s CMV12000 data should be pretty easy. The DDR4 is attached to the PS side of the ZU+, though, and the dedicated DDR controller there is shared by many elements of the system, including the ARM cores. </div>
<div>
<br /></div>
<div>
The CMV12000 front-end described above exists on the PL side. The fastest interface between the PL and the PS is a set of 128-bit AXI memory buses, exposed as the slave ports S_AXI_HPx_FPD to the PL. There are four such slave ports, but only a maximum of three can be simultaneously routed to the DDR controller:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgohOSi3ye7nrK4JA6CzaEVkQ9bK2Cpn6ZYedbqu6soSIeLbrfbBI_JKIBYinTNdCjuO2inpb8Qml-3ykBsT7YASIazJAZ6Aq7ByfZuLhRW1sZtFzxADE1rfWo-HOKlIVTi4GzkOlH7PyQ/s1600/c68.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1562" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgohOSi3ye7nrK4JA6CzaEVkQ9bK2Cpn6ZYedbqu6soSIeLbrfbBI_JKIBYinTNdCjuO2inpb8Qml-3ykBsT7YASIazJAZ6Aq7ByfZuLhRW1sZtFzxADE1rfWo-HOKlIVTi4GzkOlH7PyQ/s640/c68.png" width="624" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Up to three 128-bit AXI memory buses can be dedicated to direct PL-PS DDR access.</td></tr>
</tbody></table>
<div>
The Ultrascale+ AXI might be able to go up to 333MHz, according to the datasheet, but 250MHz is the more common setting. That's okay - that's still 96Gb/s of theoretical bus bandwidth. But you can start to see why it's infeasible to store intermediate compression data in external RAM. Even 2.5 accesses per pixel would saturate the bus.</div>
<div>
<br /></div>
<div style="text-align: left;">
For this test, I set up some custom BRAM FIFOs to use for buffering between the hard-timed pixel input and the more uncertain AXI write transfer. To keep things simple, four adjacent channels share one 64b-wide FIFO, aligning their pixel data to 16 bits. All FIFO writes happen on the px_clk when the control channel indicates valid pixel data<i>.</i></div>
<div style="text-align: left;">
<i><br /></i></div>
<div style="text-align: left;">
The other side of the FIFO is a little more confusing. I split channels 1-32 and 33-64 (8 FIFOs each) into two write groups, each with its own AXI master port with 32Gb/s of theoretical bandwidth. The bottom channels drive S_AXI_HP0_FPD and the top drive S_AXI_HP1_FPD, and I rely on the DDR controller to sort out simultaneous write requests.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEissiwMNnQbpw_zRt5Jojeloy2RTD6isFDl8mD4uHko7CDs1ljsb3FxXufKaG5xaomig7HXOhQN0RHUhaoaFUJt_67oser_COMz6vIUeezZ4pdAQyiw2ThzDnrzpW2Sj0bLwzADE-J-H1s/s1600/c69.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="464" data-original-width="1600" height="184" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEissiwMNnQbpw_zRt5Jojeloy2RTD6isFDl8mD4uHko7CDs1ljsb3FxXufKaG5xaomig7HXOhQN0RHUhaoaFUJt_67oser_COMz6vIUeezZ4pdAQyiw2ThzDnrzpW2Sj0bLwzADE-J-H1s/s640/c69.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bottom channel RAM writing test pipeline, through BRAM FIFO buffers. Top channels are similar.</td></tr>
</tbody></table>
<div style="text-align: left;">
When the FIFO levels reach a certain threshold, a write transaction is started. Each transaction is 16 bursts of 16 beats of 16 bytes, and the 16 bytes of a beat are views of the FIFO output data. For simplicity, I just alternate between views of the 8 MSBs of 16 pixels to fill each 128-bit beat. I may stick the 2 LSBs from all 64 channels in their own view at some point, but for now I can at least confirm sensor operation with the 8 MSBs.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Without further ado, the first full image off the sensor:<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhsTYkjG5383iNXolpq-kK8HVWRvPlCUiPcRQY9EspBBNhmbfCh-q957wANNP_nVpWqHQmOByZuh5K9IMWHzwtfNIVUb8c-Dxx5cD8X5aXd9dJ3zomddtd3itVJd6X-GqlAMtdZR7BGmg/s1600/c54.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1165" data-original-width="1545" height="482" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhsTYkjG5383iNXolpq-kK8HVWRvPlCUiPcRQY9EspBBNhmbfCh-q957wANNP_nVpWqHQmOByZuh5K9IMWHzwtfNIVUb8c-Dxx5cD8X5aXd9dJ3zomddtd3itVJd6X-GqlAMtdZR7BGmg/s640/c54.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">What were you expecting?</td></tr>
</tbody></table>
<div style="text-align: left;">
It turned out better than I thought, even looking like a VHS tape on rewind as it does. There are both horizontal and vertical defects. The vertical defects were concentrated in one 128px-wide column, served by a single LVDS pair, so that was easily traceable to a marginal solder joint. The horizontal defects were more likely to be missing or corrupted RAM writes. They would change position every frame.<br />
<br />
At first I suspected the DDR controller might be struggling to arbitrate between the two PL-PS ports and the ARM. The ARM might try to read program data while the image capture front-end is writing, incurring both a read/write turnaround penalty and a page change penalty. But in that case the AXI slave ports should exert back-pressure on their PL masters by deasserting the AWREADY signal, and I didn't see this happening. To further rule out ARM contention, I moved the ARM program into on-chip memory and disabled all but the two slave ports being used to write data to the DDR controller...still no good.<br />
<br />
I also tried different combinations of pixel clock speed (down to 30MHz), AXI clock speed (down to 125MHz), burst size, and total transfer size with no real change. Even with only one port writing, the problem persisted. Then I tried replacing the image views with some FIFO debug info: input/output counters and the difference used to calculate the fill level. I had expected the difference to vary up and down by one or two address units since the counters run on different clocks, but I what I saw were cases where the difference was entirely wrong, possibly triggering bad transfers.<br />
<br />
So what I had was a clock domain crossing problem. Rather than describe it in detail, I'll just link <a href="https://zipcpu.com/blog/2018/07/06/afifo.html">this article</a> that I wish I had read beforehand. The crux of it is that the <i>individual bits</i> of the counter can't be trusted to change together and if you catch them in mid-transition during an asynchronous clock overlap, you can get results that are complete nonsense, not just off-by-one. The article details a comprehensive bidirectional synchronization method using Gray code counters, but for now I just tried a simple one-way method where the input counter is driven across the clock domain with an alternating "pump" signal:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiVIzwKGQZRsjSiQSuqQiE8Mst2kuf3gwh5QN1JoB4GdJCwGyEPGODlBhzTRTKMxRmxCWl4M2L4Tay2_Lpz1n70-wo_8eVmqjuOaHFLRppcyMzvloEz8zVhMl9-aHKxNgdpTi3EcYCTXI/s1600/c70.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="366" data-original-width="1600" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiVIzwKGQZRsjSiQSuqQiE8Mst2kuf3gwh5QN1JoB4GdJCwGyEPGODlBhzTRTKMxRmxCWl4M2L4Tay2_Lpz1n70-wo_8eVmqjuOaHFLRppcyMzvloEz8zVhMl9-aHKxNgdpTi3EcYCTXI/s640/c70.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Synchronization pump for FIFO input counter.</td></tr>
</tbody></table>
<div style="text-align: left;">
The pump is driven by the LSB of the input-side counter and synchronized to the AXI clock domain through a series of flip-flops. This only works if the output-side clock is sufficiently faster than the input-side clock that it can always detect every edge of the pump signal. That's the case here, with a 250MHz axi_clk and a 60MHz px_clk. The value of <span style="font-family: "courier new" , "courier" , monospace;">in_cnt_axi</span>, the input counter pumped to the AXI clock domain, is what's compared to the output counter (which is already in the AXI clock domain) to evaluate the FIFO level and trigger AXI transfers. It's the right amount of simple for me, adding only a few flip-flops to the design.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYlvNFS1AbeYfTDjsnXn_bk7DTjIdnighJc-2VYyULG8ydnYHvz5xUN4o7TuFFVk4ogJYyytAN0s6IXpWhAFDmGxg8Jzk12CMaanz6bqGz8XxgdDNHLowsw0BW7oC_VESA0Lk5DR5GxvA/s1600/c60.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYlvNFS1AbeYfTDjsnXn_bk7DTjIdnighJc-2VYyULG8ydnYHvz5xUN4o7TuFFVk4ogJYyytAN0s6IXpWhAFDmGxg8Jzk12CMaanz6bqGz8XxgdDNHLowsw0BW7oC_VESA0Lk5DR5GxvA/s640/c60.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">And just like that, clean kerbal portraits.</td></tr>
</tbody></table>
<div style="text-align: left;">
In theory, I could read in about 170 frames this way (in 0.567s...). It currently takes me 30 seconds to get each frame off over JTAG, though, so I may want to get USB (SuperSpeed!) up and running first. More importantly, I can evaluate sensor stuff independent of the two other main challenges (wavelet pipeline and SSD sink). I'm actually surprised at the okay-ness of the raw image, but there is definitely some fixed pattern noise to contend with. I also want to try the multi-slope HDR mode, which should be great for fitting more dynamic range in the 10-bit data (with no processing on my end!).</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I started with the source and sink because, even though they're the more known tasks, they represent external constraints that are actually show-stoppers if they don't work. Now I am confident in everything up to the pixel data hand-off on the source side. The sink side is still a mess, but the hardware has been checked at least. That leaves the more unknown challenge of the wavelet compression engine. But since it's entirely built from logic, with interfaces on both ends that I control, I'm actually less worried about it. In other words, it's nice to not have to think about whether or not to build something from scratch...</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com10tag:blogger.com,1999:blog-8200098102909041178.post-30074690607456220492019-08-28T23:53:00.002-04:002019-08-30T12:27:11.499-04:00TinyCross: Electronics UpdateWhere I left off, TinyCross was at the <a href="http://scolton.blogspot.com/2019/03/tinycross-chassis-build.html">rolling chassis</a> stage. Mechanically, it went together relatively smoothly, most of the issues having been worked out in CAD. There are a few minor tweaks I'd like to make to make it lighter and narrower, but they're low priority compared to getting a first test drive in. So, on to the electronics.<br />
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqE84F1dzGcyHOeEAfz093Wmx05Gmh41UVGbHY_oLFmfcVZj76GoZVj-ljc_YoXrsEIb6691ore3vt3tbaJL1UFMdWNTGk8_r3ZzpNUCQsyd2cb1GT6K1JXcIcoBu62HjkccjqrN9MEWo/s1600/tc55.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="487" data-original-width="1600" height="194" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqE84F1dzGcyHOeEAfz093Wmx05Gmh41UVGbHY_oLFmfcVZj76GoZVj-ljc_YoXrsEIb6691ore3vt3tbaJL1UFMdWNTGk8_r3ZzpNUCQsyd2cb1GT6K1JXcIcoBu62HjkccjqrN9MEWo/s640/tc55.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">It always looks so clean until you start adding wires.</td></tr>
</tbody></table>
<div style="text-align: left;">
I've already done a post on the <a href="http://scolton.blogspot.com/2018/09/tinycross-electron-control-unit.html">motor drive design</a>. Since the kart is four wheel drive, one will control the two front motors and one will control the two rear motors. For now I've only built up one, just in case there are any observations from the first build that would require changing parts on the second. Here's what the power side looks like:</div>
</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMRj-OXGPg0AjSFMVkvKy20R9-_Bcw4JOIbCrDj-xGqv7YfD6ywUsRTK9xDKtEQZrfzYr2OdRw4NdB5UW2WWxe3cO-WS0PMZw-9GoPZ3HdWpw6cYqFri6YHIGJQSZ3lSH6jOjHY4D3gSc/s1600/tc66.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMRj-OXGPg0AjSFMVkvKy20R9-_Bcw4JOIbCrDj-xGqv7YfD6ywUsRTK9xDKtEQZrfzYr2OdRw4NdB5UW2WWxe3cO-WS0PMZw-9GoPZ3HdWpw6cYqFri6YHIGJQSZ3lSH6jOjHY4D3gSc/s640/tc66.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TxDrive, power side.</td></tr>
</tbody></table>
<div style="text-align: left;">
It's one of the weirdest power layouts I've done for a motor drive. The design supports two different FET configurations: one with a single <a href="http://ixapps.ixys.com/Datasheet/MTI200WX75GD.pdf">MTI200WX75GD</a> per motor and another with a six <a href="http://www.mouser.com/ds/2/149/FDMT80080DC-760735.pdf">FDMT80080DC</a>s. Since the MTI200's are perpetually out of stock, I committed to the FDMT solution for this build. It really doesn't look like enough FET, but on paper they're almost identical to the MTI200. I especially like the 1453A pulsed current rating. The board is four layers but only 1oz copper, so I also reinforced some of the high current density paths with 1mm bus wire and copper braid.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The FDMT 8x8 SO8 package creates a few other advantages in this configuration. The parasitic inductance is lower and there's room for local ceramic capacitor decoupling near each half bridge, which will help contain switching transients. The entire power side is also at or below 1mm in height, so the whole surface area of the board, include the somewhat overworked 12V to 5V LDO, can be heat sunk to the chassis through some thermal pad:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheNjoiUZjtM9_nx10XZfcIBIV84MfaQ4UXvsTuOuIaau9H2WZV2mnogR6vkE254IWJD5yAn2NP6fX04X41XWo3G124XPmSVytWX5Qgbm72WC2jbZHc_1WWL2WgrFDQf9ASQo7Hmis5RZc/s1600/tc62.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheNjoiUZjtM9_nx10XZfcIBIV84MfaQ4UXvsTuOuIaau9H2WZV2mnogR6vkE254IWJD5yAn2NP6fX04X41XWo3G124XPmSVytWX5Qgbm72WC2jbZHc_1WWL2WgrFDQf9ASQo7Hmis5RZc/s640/tc62.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TxDrive, signal side.</td></tr>
</tbody></table>
<div style="text-align: left;">
On the other side of the board, each half bridge gets a <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzVsW07cFFqEdOqz23OHXxmbq6V6M8z28fUkgCrfzKucUOpzXJ5QWuvQQ0dJ4APZL1WoABbt_7nPn_wWTGlwNuzwyCtNne3uPqY1jAKx4NNUn8-2fIO2zh7mkyfhD5vUTofzKiYp4_1Lw/s1600/layout09.png">12mm-wide vertical slice</a> with its phase wire exit, gate drive, current sense, and 2x47uF aluminum polymer bus capacitance. An additional 820uF of bulk capacitance per motor gets folded over into the unused volume. The signal board sits in the middle and carries the MCU, its power supply, a CAN transceiver, and the encoder interface.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Two pairs of 12AWG inputs, each with 4mm bullet connectors, support up to about 150A of peak battery current. I find it easier to deal with two 12AWG wires than one 8AWG wire. The six phase outputs are also 12AWG, so everything can pass through a common size grommet on the eventual enclosure. The only other connections are CAN (twisted pair) and the encoders (<a href="https://www.digikey.com/product-detail/en/3m/3517-9-100SF/MB09H-10-ND/1190677">9-conductor shielded ribbon cable</a>). </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The ribbon cables and phase wires run in parallel down each upper A-arm to the motors. This is the scariest run of wire for many reasons. Electrically, the phase wires are high-dV/dt EMI sources that will capacitively couple onto the encoder cable. This is the main motivation for using shielded cable and <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEher5ZXjLnQsB7UNH4-YfDcc1HvnHDFxpVLHVkrbtAdLiLGDoe-FFaoJMNk3R7eW6SiikXom7RaxHv4W8H7T2NM4uL8enZxvRruEyKWFOsMVWRZZmLwFlfrbtb8GYv1oYYbRceerpp8738/s1600/layout15.png">three-phase optoisolated</a> Hall signal configuration. Mechanically, these wires pass through several moving parts. (The encoder cable even passes <i>through</i> the drive belt loop!) They need enough slack to accommodate the entire steering and suspension travel, but the slack needs to be in the right places, with good strain relief everywhere else. The routing is actually pretty clean, and will get cleaner once the drives and encoders have their covers installed.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiDryVI1yYHNH0YyYKVx2eCSYKQAJCEHoRGDTS-ipHxu_51D2A_qNY2vgjcDktj6JC1t-8egjGR5VsnthkHT4vOUf3ehSiStbHNHsXQZeSM1Q6Tih1bI9ula8zFTZf2wnjwvicof6db8s/s1600/tc63.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhiDryVI1yYHNH0YyYKVx2eCSYKQAJCEHoRGDTS-ipHxu_51D2A_qNY2vgjcDktj6JC1t-8egjGR5VsnthkHT4vOUf3ehSiStbHNHsXQZeSM1Q6Tih1bI9ula8zFTZf2wnjwvicof6db8s/s640/tc63.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Front wheel drive fully wired up.</td></tr>
</tbody></table>
<div style="text-align: center;">
<div style="text-align: left;">
That's all just for the front drive; everything will get repeated for the rear. That means that in total there are four pairs of 12AWG DC wire to route out from the central battery input, and up to 300A of total peak battery current to deal with. And this is where I get to spread the good word about MIDI fuses. They are by far the most power-dense fuse format. I've always used the car audio ones, with questionable voltage rating, but Littelfuse makes some <a href="https://www.littelfuse.com/products/fuses/automotive-passenger-car/high-current-fuses.aspx">serious ones</a> as well, up to a 75V / 200A model rated to break 2500A! Their triple fuse holder is also perfect for my circuit.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMrLca_iMZp_J6vbFVYI7qEs1927IcoL1OPhgpLQaTYbEhkSNmz3BQ2gPjyQYHADarj-p4Y4V2ibM-Xd1QvKrgrA7VWQxLrSXAvbjQBLFtR6uumSIb5C69oCUqSHynWehnn8OPgZskOtM/s1600/tc57.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMrLca_iMZp_J6vbFVYI7qEs1927IcoL1OPhgpLQaTYbEhkSNmz3BQ2gPjyQYHADarj-p4Y4V2ibM-Xd1QvKrgrA7VWQxLrSXAvbjQBLFtR6uumSIb5C69oCUqSHynWehnn8OPgZskOtM/s640/tc57.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Main power input.</td></tr>
</tbody></table>
<div style="text-align: left;">
The two battery inputs (each a series string of two <a href="https://www.getfpv.com/tattu-plus-10000mah-22-2v-25c-6s-lipo-smart-battery-pack.html?utm_source=google&utm_medium=cpc&adpos=1o1&scid=scplp4479&sc_intid=4479&gclid=CjwKCAjwqZPrBRBnEiwAmNJsNrJksAOdqpt46B8fclg9PyWJypYmRLmFpWXvAH12PwYAQGHeJPCdYRoC4vYQAvD_BwE">Tattu 6S 10Ah Smart packs</a>) get the same dual 12AWG treatment, with back-to-back thick #10 ring terminals (the wonderful <a href="https://www.mcmaster.com/7113k17">McMaster 7113K17</a>) bolted to individual fuses.These connect to a bus bar that feeds the full 4x12AWG positive group. This goes through a master power switch and then splits off two and two to the front and rear drives. Meanwhile, a separate 30A fuse feeds off the bus bar through a small switch to the charger and steering wheel board.</div>
<h4 style="text-align: left;">
The...steering wheel board?</h4>
<div style="text-align: left;">
I am really trying to minimize the number of microcontrollers (and also the number of firmware images) on this kart. Each drive has an <a href="https://www.st.com/en/microcontrollers-microprocessors/stm32f303.html">STM32F303</a> that's pretty busy running two motors and really shouldn't be doing anything else. But I can stuff every other process onto a single high-level controller. This controller needs to handle driver interface (including throttle read-in), CAN communication with the drives, and ideally battery management. This constrains it to be somewhere near the center of the kart, and the steering wheel seemed like a logical place.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEhNag74iS7jhzSbRY_SPzdeMybUSwYpt3ATmXfP-MLZR8p7Dd8zXgZqL9wehnvZqbsSz-PPNh_U-TX5lOsvhMnXGyim4XsWHDgD2khRQg8pfVPPt8JtePrOdcLnKA7hz3du38eF-xxA4/s1600/tc65.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEhNag74iS7jhzSbRY_SPzdeMybUSwYpt3ATmXfP-MLZR8p7Dd8zXgZqL9wehnvZqbsSz-PPNh_U-TX5lOsvhMnXGyim4XsWHDgD2khRQg8pfVPPt8JtePrOdcLnKA7hz3du38eF-xxA4/s640/tc65.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Why have I not grip-taped my steering wheels before?</td></tr>
</tbody></table>
I've also always wanted to have an OLED steering wheel display. Having live data will definitely help with troubleshooting. Although it's not absolutely necessary, I decided to use the <a href="https://www.st.com/en/microcontrollers-microprocessors/stm32f7x6.html">STM32F746</a> for this board since it has the DMA2D graphics driver. The OLED is 4-bit monochrome, which isn't a natively-supported output format for the DMA2D. But as long as you're blitting even numbers of pixels, you can still make it work. The interface between the OLED and the main board is a SPI variant, good enough for a 50-60Hz update rate. I was originally going to put it on headers, but for clam shell serviceability it was better to just use thin wires.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZPAIzk6EHP2BcgF8k-UteOumTpTz3kD46waLSeXrVhjTDZV5fQWF-FDNWug6fmHlOmfuPYdwp3LtETYc2qVjfV3GNaVMe3UDRHf-9VW8u9UgrRXwcBvax0XYVF5d1q-esEbpnfz2cSss/s1600/tc64.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZPAIzk6EHP2BcgF8k-UteOumTpTz3kD46waLSeXrVhjTDZV5fQWF-FDNWug6fmHlOmfuPYdwp3LtETYc2qVjfV3GNaVMe3UDRHf-9VW8u9UgrRXwcBvax0XYVF5d1q-esEbpnfz2cSss/s640/tc64.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Display interface and "hot" side of the BMS.</td></tr>
</tbody></table>
<div style="text-align: left;">
Also on that side of the board is the battery management system (BMS) cell balance circuitry. This got out of hand quickly since I left almost no room for it: the entire area under the display is pretty much off-limits. But I managed to cram 12 cells worth of balance circuit on each side with the resistors themselves sinking heat into the steering wheel metal. To facilitate routing, the circuit alternates FET/resistor placement for the odd and even cells:</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhR9-zjvTRtmhc6wtuLEAQQD2r6jqdhce7vfZUqCsBdRUNbV7_jdx238vOzurv10gyBM2CK9P-_RBcR9VJzTTGQtJOZ1_BYcXod819sbO53CElSxfZzLWphPKhsb21Y_Vi1bVc-69YTAYk/s1600/tc68.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="906" data-original-width="1499" height="385" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhR9-zjvTRtmhc6wtuLEAQQD2r6jqdhce7vfZUqCsBdRUNbV7_jdx238vOzurv10gyBM2CK9P-_RBcR9VJzTTGQtJOZ1_BYcXod819sbO53CElSxfZzLWphPKhsb21Y_Vi1bVc-69YTAYk/s640/tc68.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Cell balance group.</td></tr>
</tbody></table>
</div>
</div>
<div style="text-align: left;">
To discharge an individual cell, a square wave is driven onto its charge pump, which turns on the its FET. This happens to the cell(s) with the highest voltage until they are evened out. Usually this is done during or after charging. During discharge, it's sufficient to just monitor the cell voltages and stop when any one cell reaches a low voltage threshold.<br />
<br />
Accurately measuring individual cell voltages is itself an interesting challenge. The main problem is that the cells are offset by up to 48V from the ADC ground. Of course, it's possible to use simple voltage dividers to bring the signals down to below 3.3V. But it would be better to have individual differential measurements of each cell. This means a lot of op-amps or...<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHQ339jy-V0sstUU101sucz8NnpVdZSsOBZNOoXVYUIWXeq-nxZcYmz-t3E5_A_GIPW2G19v1Y5JaR-SOOaR6Nfl0QeAzk9RWm2B0NojH97BS7u01USN1YjuH5pdZIenenqNRJ9preJ8A/s1600/tc69.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="515" data-original-width="1600" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHQ339jy-V0sstUU101sucz8NnpVdZSsOBZNOoXVYUIWXeq-nxZcYmz-t3E5_A_GIPW2G19v1Y5JaR-SOOaR6Nfl0QeAzk9RWm2B0NojH97BS7u01USN1YjuH5pdZIenenqNRJ9preJ8A/s640/tc69.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">One op-amp and a 72V analog multiplexer for cell voltage measurement.</td></tr>
</tbody></table>
<div style="text-align: left;">
I found some 72V analog multiplexers (<a href="https://datasheets.maximintegrated.com/en/ds/MAX14752-MAX14753.pdf">MAX14753</a>) that can feed the inputs of one nice op-amp. The muxes are dual 4-to-1 selectors cascaded and wired such that the two outputs are always adjacent cell nodes, which drive the inputs of a differential amplifier. This all fits in a pretty small footprint on the opposite side of the board from the cell balance circuitry. Also on this side of the board are all the connectors, the logic and analog power supplies, the charge cutoff FETs, buffers for driving the cell balance charge pumps, a very sad SD card holder with a reversed footprint, the STM32F7 itself, and a mystery component.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjso042IyQl1aa1VPEShpX5h1Bbdgls4hpMuP6FVRhdrY13OHqfDv7hx5q34i12a4UmJn_eqNT_NhDqc1oziFJMIgsCoHBdJLwvE2xcV2CTrJT8ROc8veyH1IX27wZdoZACoJQGn4Cvoh8/s1600/tc58.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjso042IyQl1aa1VPEShpX5h1Bbdgls4hpMuP6FVRhdrY13OHqfDv7hx5q34i12a4UmJn_eqNT_NhDqc1oziFJMIgsCoHBdJLwvE2xcV2CTrJT8ROc8veyH1IX27wZdoZACoJQGn4Cvoh8/s640/tc58.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The crowded side of the steering wheel board.</td></tr>
</tbody></table>
<div style="text-align: left;">
Right now the main purpose of this board is to act as the high-level controller for commanding torque and reading back data from the motor drives. The BMS functionality is a secondary objective, since I can still monitor pack voltage through the drives and charge off-board. The torque command comes from a nice trigger stolen from <a href="https://www.amazon.com/FS-GT2-Channel-Digital-Transmitter-Receiver/dp/B00KHLDCX4/ref=sr_1_4?keywords=RC+car+transmitter&qid=1567003056&s=gateway&sr=8-4">Amazon's second cheapest RC car transmitter</a>. Like <a href="http://scolton.blogspot.com/p/cap-kart.html#tinykart">tinyKart</a>, this means all the controls are on the steering wheel - no pedals. The trigger is bidirectional, so it can command positive and negative torque. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
All four motors receive a torque command over CAN at 1kHz that they apply to their current controllers. The motors then take turns replying with their crucial data (electrical angle, speed, voltage, current, and fault status) at 250Hz, and their less important data at 50Hz. This should allow for some fairly tight feedback loops through the central controller for things like speed control, traction control, and torque vectoring. There's also that mystery component, which is controls-related.<br />
<br />
For now, I'm just starting to test the power system with the front drive only, quite honestly so I can see the fire when it happens. The two motors will just get the same torque command, ramping up slowly to full voltage/current. I did as much testing as I could on power supplies, but it's finally time for batteries. Here's the first batteries-in test, at a very easy 6V/20A (peak line-to-neutral quantities) limit:<br />
<br />
<div style="text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/WfxM44ryrbA" width="640"></iframe><br /></div>
<br />
It's nothing exciting, but the first batteries-in test is always a bit scary since there's no longer a CC/CV supply keeping things from going out of hand. After I do some wiring and software clean up and make sure the data logging is working, I'll ramp up from there toward the full 24V/120A, and then full four wheel drive. I've learned to expect smoke at some point during this process though, so I'm holding off on building the second drive until I see what fails on the first...</div>
</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com10tag:blogger.com,1999:blog-8200098102909041178.post-54257744882861858472019-07-15T14:43:00.000-04:002019-09-04T22:58:14.680-04:00Benchmarking NVMe through the Zynq Ultrascale+ PL PCIe Linux Root Port DriverI want to be able to <a href="http://scolton.blogspot.com/2019/06/freight-train-of-pixels.html">sink 1GB/s</a> into an NVMe SSD from a Zynq Ultrascale+ device, something I know is technically possible but I haven't seen demonstrated without proprietary hardware accelerators. The software approach - through Linux and the Xilinx drivers - has enough documentation scattered around to make work, if you have a lot of patience. But the only speed reference I could find for it is this <a href="http://www.fpgadeveloper.com/2016/07/measuring-the-speed-of-an-nvme-pcie-ssd-in-petalinux.html">Z-7030 benchmark</a> of 84.7MB/s. I found nothing for the newer ZU+, with the XDMA PCIe Bridge driver. I wasn't expecting it to be fast enough, but it seemed worth the effort to do a speed test.<br />
<div>
<br /></div>
<div>
For hardware, I have my TE0803 carrier with the <a href="https://shop.trenz-electronic.de/en/TE0803-02-04CG-1EA-MPSoC-Module-with-Xilinx-Zynq-UltraScale-ZU4CG-1E-2-GByte-DDR4-5.2-x-7.6-cm">ZU4CG version of the TE0803</a>. All I need for this test is JTAG, UART, and the four PL-side GT transceivers, for PCIe Gen3 x4. I made a JTAG + UART cable out of the <a href="https://www.digikey.com/product-detail/en/digilent-inc/410-357-B/1286-1196-ND/8605091">Digilent combo part</a>, which is directly supported in Vivado and saves a separate USB port for the terminal. Using Trenz's bare board files, it was pretty quick to set up.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB8kNE6wkE61Xydq45DiHttb50qkdQ43kJVxwWpNpw1hH7qa8MCuXzgx3XGeFNADZ_vWtvY8-oK_P-Y4cV9BLC2zf_clKrhUWNFaEmPgXUu5pD9lDv5vqs9X3ed91jQlAlC7JCF5rxo1A/s1600/c19.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB8kNE6wkE61Xydq45DiHttb50qkdQ43kJVxwWpNpw1hH7qa8MCuXzgx3XGeFNADZ_vWtvY8-oK_P-Y4cV9BLC2zf_clKrhUWNFaEmPgXUu5pD9lDv5vqs9X3ed91jQlAlC7JCF5rxo1A/s640/c19.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TE0803 Carrier + Dead-Bugged Digilent JTAG+USB Adapter.</td></tr>
</tbody></table>
<div style="text-align: left;">
Next, I wanted to validate the PCIe routing with a loopback test, following <a href="https://www.youtube.com/watch?v=qqi5ohBa-EY">this video</a> as a guide. I made my own loopback out of the <a href="https://www.amazon.com/EXPLOMOS-NGFF-Adapter-Power-Cable/dp/B074Z5YKXJ">cheapest M.2 to PCIe x4 adapter Amazon has to offer</a> by desoldering the PCIe x4 connector and putting in twisted pairs. This worked out nicely since I could intentionally mismatch the length of one pair to get a negative result, confirming I wasn't in some internal loopback mode.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8yuMY7jouxtyw6Ed2j-XMWIVZc5JMkkMF1cmGypC60qCrtWBAQTq9PX545y4z-bcP_Tvi1dvBLs5F4he7HNmPSi-4bjPd6hE7zozn-7uWaIhvHfi9XsItpDSqfIM_-FyTEYAMDQE_bJk/s1600/c23.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8yuMY7jouxtyw6Ed2j-XMWIVZc5JMkkMF1cmGypC60qCrtWBAQTq9PX545y4z-bcP_Tvi1dvBLs5F4he7HNmPSi-4bjPd6hE7zozn-7uWaIhvHfi9XsItpDSqfIM_-FyTEYAMDQE_bJk/s640/c23.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The three-eyed...something.</td></tr>
</tbody></table>
<div style="text-align: left;">
For most of the rest of this test, I'm roughly following the script from the <a href="https://github.com/fpgadeveloper/fpga-drive-aximm-pcie">FPGA Drive example design</a> readme files, with deviations for my custom board and for Vivado 2019.1 support. The scripts there generates a Vivado project and block design with the Processing System and the XDMA PCIe Bridge. I had a few hardware differences that had to be taken care of manually (EMIO UART, inverted SSD reset signal), but having a reference design to start from was great.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The example design includes a standalone application for simply checking that a PCIe drive enumerates on the bus, but it isn't built for the ZU+. As the readme mentions, there had been no standalone driver for the XDMA PCIe Bridge. Well, as of Vivado 2019.1, there is! In SDK, the standalone project for <b>xdmapcie_rc_enumerate_example.c</b> can be imported directly from the peripheral driver list in <b>system.mss</b> from the exported hardware.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCiL9H_aBZoG4tnYsIkr6MZoljEzQoT_Vc9gAWn171L6HwD-0uyY9TjB9JmKXMTeiFWAT6OdGgt92H94w16hXOOfbY0i5dzBpVFl48HaI4x_VP-bppxxLFMvV_vJcqMfErIPtdthtfghE/s1600/c27.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="647" data-original-width="1366" height="302" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCiL9H_aBZoG4tnYsIkr6MZoljEzQoT_Vc9gAWn171L6HwD-0uyY9TjB9JmKXMTeiFWAT6OdGgt92H94w16hXOOfbY0i5dzBpVFl48HaI4x_VP-bppxxLFMvV_vJcqMfErIPtdthtfghE/s640/c27.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">XDMA standalone driver example is available as of Vivado 2019.1!</td></tr>
</tbody></table>
<div style="text-align: left;">
I installed an SSD and ran this project and much to my amazement, the enumeration succeeded. By looking at the PHY Status/Control register at offset 0x144 from the Bridge Register Memory Map base address (0x400000000 here), I was also able to confirm that link training had finished and the link was Gen3 x4. (Documentation for this is in <a href="https://www.xilinx.com/support/documentation/ip_documentation/axi_pcie3/v3_0/pg194-axi-bridge-pcie-gen3.pdf">PG194</a>.) Off to a good start, then.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvhI7MqNy1asaAt7JP0qE8M5uDKuZ1iQyJDHmhylX4JevSs0C2LAHEcDPMtr4v_x_1RhzHcx3KGqtb6D6Gnv_859Fg9fcLYnlXk3xEy-0jk0ZXHoYIsGrRT5B4abE7HzZEZXTGF2YsGCA/s1600/c32.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvhI7MqNy1asaAt7JP0qE8M5uDKuZ1iQyJDHmhylX4JevSs0C2LAHEcDPMtr4v_x_1RhzHcx3KGqtb6D6Gnv_859Fg9fcLYnlXk3xEy-0jk0ZXHoYIsGrRT5B4abE7HzZEZXTGF2YsGCA/s640/c32.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Installed a 1TB Samsung 970 EVO Plus.</td></tr>
</tbody></table>
Unfortunately, that's where the road seems to end in terms of quick and easy setup. The next stage involves PetaLinux, which is a toolchain for building the Xilinx Linux kernel. I don't know about other people, but every time the words "Linux" and "toolchain" cross my path, I automatically lose a week of time to setup and debugging. This was no exception.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Unsurprisingly, PetaLinux tools run in Linux. I went off on a bit of a tangent trying to see if they would run in <a href="https://devblogs.microsoft.com/commandline/wsl-2-is-now-available-in-windows-insiders/">WSL2</a>. They do, if you contain your project in the Linux file system. In other words, I couldn't get it to work on /mnt/c/... but it worked fine if the project was in ~/home/... But, WSL2 is a bit bleeding edge still and there's no USB support as of now. So you can build, but not JTAG boot. If you boot from an SD card, though, it might work for you.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
So I started over with a <a href="https://www.virtualbox.org/">VirtualBox VM</a> running Ubuntu 18.04, which was mercifully easy to set up. For reasons I cannot at all come to terms with, you need at least 100GB of VM disk space for the PetaLinux build environment, all to generate a boot image that measures in the 10s of MB. I understand that tools like this tend to clone in entire repositories of dependencies, but seriously?! It's larger than all of my other development tools combined. I don't need the entire history of every tool involved in the build...</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSy2kEfBDczPUY8nIWd7QIrD3qwLp4_4veY21HuACNkQuZu8a8OcsO9VcD94WdS-d6Z-3L0encvTHHomwzaMP3xy1HVjS46Ylf2iOKHLs_AWFi9aApF94MtZ5MeNq6cRfno7kFUeRQvxc/s1600/c28.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="652" data-original-width="1600" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSy2kEfBDczPUY8nIWd7QIrD3qwLp4_4veY21HuACNkQuZu8a8OcsO9VcD94WdS-d6Z-3L0encvTHHomwzaMP3xy1HVjS46Ylf2iOKHLs_AWFi9aApF94MtZ5MeNq6cRfno7kFUeRQvxc/s640/c28.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">And here I thought Xilinx was a disk space hog...</td></tr>
</tbody></table>
<div style="text-align: left;">
The build process, even not including the initial creation of this giant environment, is also painfully slow. If you are running it on a VM, throw as many cores at it as you can and then still plan to go do something else for an hour. I started from the build script in the FPGA Drive example design, making sure it targetted <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">cpu_type="zynqMP"</span> and <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">pcie_ip="xdma"</span>. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
This <i>should</i> set up the kernel properly, but some of the config options in PetaLinux 2019.1 might not exactly match the packaged configs. There's a reference <a href="http://www.fpgadeveloper.com/2016/04/conne">here</a> explaining how to manually configure the kernel for PCIe and NVMe hosting on the Z-7030. I went through that, subbing in what I thought were correct ZU+ and XDMA equivalents where necessary. Specifically:</div>
<div style="text-align: left;">
</div>
<ul>
<li>It seems like as of PetaLinux 2019.1 (v4.19.0), there's an entire new menu item under <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Bus support</span> for <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">PCI Express Port Bus support</span>. Including this expands the menu with other PCI Express-specific items, which I left at whatever their default state was.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_-JlH6Qu5hwLeCvVDSKi65iTffSyMh-mNkr2e0oGLYCN-5kVDFWVCeClGRmEvfMZD12ycJ6RflzzquNYntPCUE8ecELdjsxJh1BQoxQYbxLkyhgG6-sa66Q518KiJe1WIkSN1672qtiw/s1600/c29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="423" data-original-width="710" height="237" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_-JlH6Qu5hwLeCvVDSKi65iTffSyMh-mNkr2e0oGLYCN-5kVDFWVCeClGRmEvfMZD12ycJ6RflzzquNYntPCUE8ecELdjsxJh1BQoxQYbxLkyhgG6-sa66Q518KiJe1WIkSN1672qtiw/s400/c29.png" width="400" /></a></div>
<ul>
<li>Under <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Bus support > PCI controller drivers</span>, <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Xilinx XDMA PL PCIe host bridge support</span> has to be included. I don't actually know if the <span style="color: #b6d7a8;"><span style="font-family: "courier new" , "courier" , monospace;">NWL PCIe Core</span> </span>is also required, but left it in since it was enabled by default. It might be the driver for the PS-side PCIe?</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjm0kyHvFL70KH9EqHG5yGdefMtB-RDIMsFjAHF4sMY51o46vzl14ldDRkfjN8wz4nQiakd97WqiTjQTjwoSIuGCIrNQN-xfhOJF735iuVjEtBohe3e3OJieoFn44yR95PF3fShePVncg0/s1600/c30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="428" data-original-width="712" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjm0kyHvFL70KH9EqHG5yGdefMtB-RDIMsFjAHF4sMY51o46vzl14ldDRkfjN8wz4nQiakd97WqiTjQTjwoSIuGCIrNQN-xfhOJF735iuVjEtBohe3e3OJieoFn44yR95PF3fShePVncg0/s400/c30.png" width="400" /></a></div>
<div>
<ul>
<li>Some things related to NVMe are in slightly different places. There's an item called <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Enable the block layer</span> on the main config page that I assume should be included. Under <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Device Drivers</span>, <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Block devices</span> should be included. And under <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">Device Drivers > NVME Support</span>, <span style="color: #b6d7a8; font-family: "courier new" , "courier" , monospace;">NVM Express block device</span> should also be included.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhemQNOWI0zBCOipiUtV-w_pIf_-12Fd53zxSwXncu1AlswaWbz7J1Xr4Uckc5gZGmqT7TLycMqxCQ_A8bZ99EJJhAJeTE9a2an5aHF5HfODqRZgVFBLUdh-wW3TmKC5Qo4tXcEurIEM2A/s1600/c31.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="424" data-original-width="718" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhemQNOWI0zBCOipiUtV-w_pIf_-12Fd53zxSwXncu1AlswaWbz7J1Xr4Uckc5gZGmqT7TLycMqxCQ_A8bZ99EJJhAJeTE9a2an5aHF5HfODqRZgVFBLUdh-wW3TmKC5Qo4tXcEurIEM2A/s400/c31.png" width="400" /></a></div>
<div style="text-align: center;">
<br /></div>
</div>
<div style="text-align: left;">
The rest of the kernel and rootfs config seems to match pretty closely the Z-7030 setup linked above. But I will admit it took me three attempts to create a build that worked, and I don't know exactly what trial-and-error steps I did between each one. Even once the correct controller driver (<b>pcie-xdma-pl.c</b>) was being included in the build, I couldn't get it to compile successfully without <a href="https://github.com/Xilinx/linux-xlnx/commit/bc110c1e7da48439835265a0fbe9f8fc57cad752#diff-df396aeff136bd2a390eaa1fe1be8639">this patch</a>. I don't know what the deal is with that, but after that I finally got a build that would enumerate the SSD on the PCIe bus:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg54JEc_WGiHJY0fI6CA9av67TfVK_Se9gLdIc5scKE1i-s6hqXOonlzSrHsZg_WaFnIHUb9CwlcZQTvC9ALSu3fSuFczLGex87hIKPmRvfRVhsm9GU-2cO2p5y12QNTuU9tdlhk7ra4fc/s1600/c33.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="414" data-original-width="877" height="302" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg54JEc_WGiHJY0fI6CA9av67TfVK_Se9gLdIc5scKE1i-s6hqXOonlzSrHsZg_WaFnIHUb9CwlcZQTvC9ALSu3fSuFczLGex87hIKPmRvfRVhsm9GU-2cO2p5y12QNTuU9tdlhk7ra4fc/s640/c33.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Output from <b>lspci -vv </b>confirms link speed and width.</td></tr>
</tbody></table>
<div style="text-align: left;">
I had already partitioned the drive off-board, so I skipped over those steps and went straight to the speed tests as described <a href="http://www.fpgadeveloper.com/2016/07/measuring-the-speed-of-an-nvme-pcie-ssd-in-petalinux.html">here</a>. I tested a few different block sizes and counts with pretty consistent results: about <span style="color: yellow;">460MB/s write</span> and <span style="color: yellow;">630MB/s read</span>.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_evwsZPn5FJWsAHL2dcct2l5ERD8DJukiU7jIGDaYWE0C1loY9E_kQ-H3WlhRr_25m5kjYI6-IRM5ycplxNfuisjnhA0QGpaOLb9mKXOt1oxVBYitR-ud163zmCUg2ZrJHjhNr0j9980/s1600/c35.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="574" data-original-width="1275" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_evwsZPn5FJWsAHL2dcct2l5ERD8DJukiU7jIGDaYWE0C1loY9E_kQ-H3WlhRr_25m5kjYI6-IRM5ycplxNfuisjnhA0QGpaOLb9mKXOt1oxVBYitR-ud163zmCUg2ZrJHjhNr0j9980/s640/c35.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Not sure about those correctable errors. I guess it's better than un-correctable errors.</td></tr>
</tbody></table>
<div style="text-align: left;">
That <i>is</i> actually pretty fast, compared to the Z-7030 benchmark. The ZU+ and the new driver seem like they're able to make much better use of the SSD. But, it's still about a factor of two below what I want. There could be some extra performance to squeeze out from driver optimization, but at this point I feel like the effort will be better-spent looking into hardware acceleration, which <a href="https://www.youtube.com/watch?time_continue=34&v=ivcm2nwGsQM">has been demonstrated</a> to get to 1GB/s write speeds, even on older hardware.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Since there's no published datasheet or pricing information for that or any other NVMe hardware accelerator, I'm not inclined to even consider it as an option. At very least, I plan to read through the open specification and see what actually is required of an NVMe host. If it's feasible, I'd definitely prefer an ultralight custom core to a black-box IP...but that's just me. In the mean time, I have some parallel development paths to work on.</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com26tag:blogger.com,1999:blog-8200098102909041178.post-14893074330469405112019-06-13T01:28:00.000-04:002020-04-18T22:44:30.864-04:00Freight Train of PixelsI have a problem. After any amount of time at any level of development of anything, I feel the urge to move down one layer into a place where I really shouldn't be. Thus, after spending time implementing capture software for my <a href="http://scolton.blogspot.com/p/video.html#gs3"><strike>Point Grey</strike> FLIR block cameras</a>, I am now tired of dealing with USB cables and drivers and firmware and settings.<br />
<br />
What I want is image data. Pixels straight from a sensor. As many as I can get, as fast as I can get them. To quote Jeremy Clarkson from The Great Train Race (<a href="https://www.topgear.com/videos/top-gear-tv/great-train-race-part-14-series-13-episode-1">Top Gear S13E1</a>), "Make millions of coals go in there." Except instead of coals, pixels. And instead of millions, trillions. It doesn't matter how. I mean, it does, but really I want the only real constraints to be where the pixels are coming from and where they are going. So let's see what's in this rabbit hole.<br />
<h4>
The Source</h4>
<div>
The image sensor feeding this monster will be an ams (formerly CMOSIS) <a href="https://ams.com/cmv12000">CMV12000</a>. It's got lots of pros and a few cons for this type of project, which I'll get into in more detail. But the main reason for the choice is entirely non-technical: This is a sensor that I can get a <a href="https://ams.com/cmv12000#tab/documents">full datasheet</a> for and <a href="https://ams.com/cmv12000#tab/shop-now">purchase</a> without any fucking around. This was true even back in the CMOSIS days, but as an active ams part it's now documented and distributed the same way as their $1 ICs.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioCGko5my065A3wUT8ydaEnf7TovVid40ES864RwE1S-74UjaxQxlwcgjTJ7xvD1RiIsg048BWP4ZVnyTii2RwjDUbCJFqmN1MN3GdmKjFOtQiZWAM6q-aGiYJjIVbVKLhISgBPUgpbrE/s1600/c17.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1026" data-original-width="1600" height="410" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioCGko5my065A3wUT8ydaEnf7TovVid40ES864RwE1S-74UjaxQxlwcgjTJ7xvD1RiIsg048BWP4ZVnyTii2RwjDUbCJFqmN1MN3GdmKjFOtQiZWAM6q-aGiYJjIVbVKLhISgBPUgpbrE/s640/c17.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The CMV12000 is not $1, sadly, but you can Buy It Now if you really want. For prototyping, I have two monochrome ones that came from a heavily-discounted surplus listing. Hopefully they turn on.</td></tr>
</tbody></table>
</div>
<div>
This is a case, then, where the available component drives the design. The CMV12000 is <i>not</i> going to win an image quality shootout with a 4K Sony sensor, but it <i>is</i> remarkably fast for its resolution: up to <span style="color: yellow;">300fps at 4096x3072</span>. That's 3.8Gpx/s, somewhere between the total camera interface on a <a href="https://youtu.be/Ucp0TTmvqOE?t=4859">Tesla Full Self Driving Chip</a> (2.5Gpx/s) and the imaging rate of a <a href="https://www.phantomhighspeed.com/products/cameras/4kmedia/flex4k">Phantom Flex 4K</a> (8.9Gpx/s). There was a <a href="https://ams.com/documents/20143/36005/CMV12000_AN000462_1-00.pdf">version jump</a> on this sensor that I think moved it into a different category of speed, and that's where I'm placing the lever for pushing this design.</div>
<div>
<br /></div>
<div>
The CMV12000 is also a global shutter CMOS sensor, something more common in industrial and machine vision applications than consumer cameras. The entire frame is sampled at once, instead of row-by-row as in rolling shutter CMOS. (The <a href="https://www.youtube.com/watch?v=nP1elMR5qjc">standupmaths</a> video on the topic is my favorite.) The advantage is that moving objects and camera panning don't create distortion, which is arguably just correct behavior for an image sensor... But although a few pro cameras with global shutter have existed, even those have mostly died out. This is due to an interlinked set of trade-offs that give rolling shutter designs the advantage in cost and/or dynamic range.</div>
<div>
<br /></div>
<div>
For engineering applications, though, a global shutter sensor with an external trigger is essentially a visual oscilloscope, and can be useful beyond just creating normal video. By synchronizing the exposure to a periodic event, you can measure frequencies or visualize oscillations well beyond the frame rate of the sensor. Here's an example of my global shutter Grasshopper 3 camera capturing the cycle of a pixel shifting DLP projector. Each state is 1s/720 in duration, but the trigger can be set to any multiple of that period, plus or minus a tiny bit, to capture the sequence with an effective frame rate much higher than 720fps.</div>
<div>
<br /></div>
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/YtBDJjSPTlo" width="640"></iframe>
<br />
<div>
<br /></div>
<div>
Whether a consequence of the global shutter or not, the main on-paper shortcoming of the CMV12000 is the relatively high dark noise of <span style="color: red;">13e-</span>. For comparison, the <a href="https://www.sony-semicon.co.jp/products_en/new_pro/may_2017/imx294cjk_e.html">Sony IMX294CJK</a>, the 4K sensor in some new cameras with very good low-light capability, is below 2e-. That's a rolling shutter sensor, though. Sony also makes low-noise global shutter CMOS sensors like the <a href="https://www.sony-semicon.co.jp/products_en/IS/sensor0/img/product/cmos/IMX253_255LLR_LQR_Flyer.pdf">IMX253</a>, at around 2.5e-. The extra noise on the CMV12000 will mean that it needs more light for the same image quality compared to these sensors.</div>
<div>
<br /></div>
<div>
Even given adequate light, the higher noise also eats into the dynamic range of the sensor. The signal-to-noise ratio for a given saturation depth will be lower. This means either noisy shadows or blown-out highlights. But the CMV12000 has a feature I haven't seen on any other commercially-available sensor: a per-pixel stepped partial reset. The theory is to temporarily stop accumulating charge on bright pixels when they hit intermediate voltages, while allowing dark pixels to keep integrating. Section 4.5.1 in <a href="https://ora.ox.ac.uk/objects/uuid:a97cabab-5058-4267-9a0c-559d40af300a">this thesis</a> has more on this method.<br />
<br />
In the example below, the charge reading is simulated for 16 stops of contrast. With baseline lighting, the bottom four stops are lost in the noise and the top four are blown out. Increasing the illumination by 4x recovers two stops on the bottom, but loses two on top. The partial reset capability slows down the brightest pixels, recovering several more stops on top without affecting the dark pixels. The extra light is still needed to overcome the dark noise, but it's less of an issue in terms of dynamic range.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgg6ol4BHTWgXQ7Fmmr8LvrtZeCcvhMiZMvsMwO420yy8QCjGipO05e4PK-wMFOQa21KKf1LSuysZ4-Y4v7433zZCtFLm0x_hsvG3xDG5CvrosojeclkOPi1dbniVFve9RuUusG0alYeZU/s1600/c08.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1079" data-original-width="1600" height="430" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgg6ol4BHTWgXQ7Fmmr8LvrtZeCcvhMiZMvsMwO420yy8QCjGipO05e4PK-wMFOQa21KKf1LSuysZ4-Y4v7433zZCtFLm0x_hsvG3xDG5CvrosojeclkOPi1dbniVFve9RuUusG0alYeZU/s640/c08.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Dynamic range recovery using 3-stage partial reset.</td></tr>
</tbody></table>
<div style="text-align: left;">
The end result of partial reset is a non-linear pixel response to illumination. This is often done anyway, after the ADC conversion, to create log formats that compress more dynamic range into fewer bits per pixel. Having hardware that does something similar in-pixel, before the ADC, is a powerful feature that's not at all common.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Another aspect of the CMV12000 that helps with implementation is the pixel data interface: the data is spread out on 64 parallel LVDS output pairs that each serve a group of pixel columns. This extra-wide bus means more reasonable clock speeds: 300MHz DDR (600Mb/s) for full rate. A half-meter wavelength means wide intra-pair routing tolerances. There is still a massive 4.8ns inter-channel skew that has to be dealt with, but it would be futile to try to length match that. The sensor does put out training data meant for synchronizing the individual channels at the receiver, which is a headache I plan to have in the future.</div>
<div style="text-align: left;">
<h4>
The Sink</h4>
</div>
<div style="text-align: left;">
I'm starting from the assumption that it's impossible to really do anything permanent with 38Gb/s of data, if you're working with hardware at or below that of a laptop PC. In an early concept, I was planning to just route the data to a PCIe x4 output and send it in to something like an Intel NUC for further processing. But even that isn't fast enough for the CMV12000. (Also, you can buy <a href="https://www.ximea.com/en/products/xilab-application-specific-custom-oem/embedded-vision-and-multi-camera-setup-xix/cmosis-cmv12000-color-4k-embedded-camera">something like that</a> already. No fun.) And even if you could set up a 40Gb/s link to a host PC through something like Thunderbolt 3, it's really just kicking the problem down the road to more and more general hardware, which probably means more Watts per bit per second.<br />
<br />
Ultimately, unless the data is consumed immediately (as with a machine vision algorithm that uses one frame and then discards it), or buffered into RAM as a short clip (as with circular buffers in high-speed cameras), the only way to sink this much data reasonably is to compress it. <span style="color: white;">And this is where this project goes off the rails a little.</span><br />
<br />
For starters, I choose <span style="color: yellow;">1GB/s</span> as a reasonable sink rate for the data. This is within reach of NVMe SSD write speeds, and makes for completely reasonable recording times of 17min/TB (at maximum frame rate). This is very light compression, as far as video goes - less than <span style="color: yellow;">5:1</span>. I think the best tool for the job is probably wavelet compression, rather than something like h.265. It's intra-frame and uses relatively simple logic, which means fast and cheap. But putting aside the question of how fast and how cheap for now, I first just want to make sure the quality would be acceptable.<br />
<br />
There are several good examples of wavelet compression already in use: <a href="http://www.cs.tut.fi/~tabus/course/SC/246pagesCourseonJPEG2000.pdf">JPEG2000</a> uses different variants for lossless and lossy image compression. <a href="https://www.red.com/red-101/redcode-file-format">REDCODE</a> is wavelet-based and 5:1 is a standard setting described as "visually lossless". <a href="https://github.com/gopro/cineform-sdk">CineForm</a> is a wavelet codec recently open-sourced by GoPro. The SDK for CineForm includes a lightweight example project that just compresses a monochrome image with different settings. Running a test image through that with settings close to 5:1 produces good results:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXl9eA5ZoIjN73M8fRmmM-SMtcYw6lU89Kx2CfdjSWgMa1d9T9AFf1Vh5DRfGQfPRDL6Tv1MhyphenhyphenrMs2QMbKvVyx5FDSa38LFJzw8Nk8blt-_VLuw03F-SfniDy_D7YgSGKgy5NMpW0ixAQ/s1600/c12.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXl9eA5ZoIjN73M8fRmmM-SMtcYw6lU89Kx2CfdjSWgMa1d9T9AFf1Vh5DRfGQfPRDL6Tv1MhyphenhyphenrMs2QMbKvVyx5FDSa38LFJzw8Nk8blt-_VLuw03F-SfniDy_D7YgSGKgy5NMpW0ixAQ/s640/c12.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The original monochrome image.</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-s-tSoQ8me77sJs0pONI0LIb77zr5ash1kkkSSZiqgpVn6VSK7lN4BOTY6zAHs2Evd4So5BF4Wjv7FExnpnsh01GtyiVN2xvmnKruRspFSDj50Tm6_9Fd1fWFCNuYotALqKfwsrhpJS0/s1600/c13.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-s-tSoQ8me77sJs0pONI0LIb77zr5ash1kkkSSZiqgpVn6VSK7lN4BOTY6zAHs2Evd4So5BF4Wjv7FExnpnsh01GtyiVN2xvmnKruRspFSDj50Tm6_9Fd1fWFCNuYotALqKfwsrhpJS0/s640/c13.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The wavelet transform outputs a 1/8-scale low-frequency thumbnail and three stages of quantized high-frequency blocks, which are sparse and easy to compress. I just zipped this image as a test and got a 5.7:1 ratio with these settings.</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguyoN7Y1SB1yx3IIUe2unr1VCyCk8TDq1vocvmPHrI_Osc9xy2InftgIcYoEiBnmoukdKsUR9AaBeY69OKKj6W3LAs-YTyuEePuzc4kizJPIiGnbcgQgrgxIPlXe0ULdCF0vf0_CqNcTo/s1600/c14.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguyoN7Y1SB1yx3IIUe2unr1VCyCk8TDq1vocvmPHrI_Osc9xy2InftgIcYoEiBnmoukdKsUR9AaBeY69OKKj6W3LAs-YTyuEePuzc4kizJPIiGnbcgQgrgxIPlXe0ULdCF0vf0_CqNcTo/s640/c14.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The recovered image.</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUmgdnxS2amoYUG1lxKVRiVXf2a78bdAPKsif23oYucbHm9SwC9BXAkKzzIfk9MMO5BNJJYR1Bip6arH0sjV9NAacyDCED2gWGju7YKuODGMmOpnyLe6oG1htmzpbzIbl8nGVBoXHtgVM/s1600/c09.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="825" data-original-width="1166" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUmgdnxS2amoYUG1lxKVRiVXf2a78bdAPKsif23oYucbHm9SwC9BXAkKzzIfk9MMO5BNJJYR1Bip6arH0sjV9NAacyDCED2gWGju7YKuODGMmOpnyLe6oG1htmzpbzIbl8nGVBoXHtgVM/s640/c09.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Since these images are going to be destroyed by rescaling anyway, here's a 400% zoom of some high-contrast features.</td></tr>
</tbody></table>
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUmgdnxS2amoYUG1lxKVRiVXf2a78bdAPKsif23oYucbHm9SwC9BXAkKzzIfk9MMO5BNJJYR1Bip6arH0sjV9NAacyDCED2gWGju7YKuODGMmOpnyLe6oG1htmzpbzIbl8nGVBoXHtgVM/s1600/c09.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a><br /></div>
The choice of wavelet type does matter, but I think the quantization strategy is even more important. The wavelet transform doesn't reduce the size of the data, it just splits it into low-frequency and high-frequency blocks. In fact, for all but the simplest wavelets, the blocks require more bits to store than the original pixels:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibzOEzCS3XjYcnUdOPCNwEv6zNbrM6w8x4_NfRC2eTISI81HnSz6NVzYMK2ZHipCG89Exl6H5KRoWJFSNxQF9l8zimdwmEHvkMl9uBk0J0dKMuRARSbGEigtnFzJAgW2cpkaVsg83WPB0/s1600/wv02.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1154" data-original-width="1430" height="515" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibzOEzCS3XjYcnUdOPCNwEv6zNbrM6w8x4_NfRC2eTISI81HnSz6NVzYMK2ZHipCG89Exl6H5KRoWJFSNxQF9l8zimdwmEHvkMl9uBk0J0dKMuRARSbGEigtnFzJAgW2cpkaVsg83WPB0/s640/wv02.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Output range maps for different wavelets. All but the simplest wavelets (Haar, Bilinear) have corner cases of low-frequency or high-frequency outputs that require one extra bit to store.</td></tr>
</tbody></table>
<div style="text-align: center;">
<div style="text-align: left;">
<div style="text-align: left;">
Take the Cineform 2/6 wavelet (a.k.a <a href="http://wavelets.pybytes.com/wavelet/rbio1.3/">reverse biorthogonal 1.3</a>?) as an example: the low-frequency block is just an average of two adjacent pixels, so it doesn't need any more bits than the source data. But the high-frequency blocks look at six adjacent pixels and could, for some corner cases, give a result that's larger than the maximum pixel amplitude. It needs one extra bit to store the result without clipping. Seems like we're going in the wrong direction!<br />
<br /></div>
<div style="text-align: left;">
</div>
<div style="text-align: left;">
Like most image compression techniques, the important fact is that the high frequency information is less valuable, and can be manipulated and even discarded without as much visual penalty. By applying a deadband and quantization step to the high-frequency blocks, the data becomes more sparse and easier to compress. Since this is the lossy part of the algorithm, the details are hugely important. I have a little sandbox program that I use to play with different wavelet and quantization settings on test images. In most cases, 5:1 compression is very reasonable.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqb714dnLd4qKZw6Zif5ovZyOxLxM8dm_hqrvPB1gDHmHZSConeGdvRB1R27PIXNoVHTl8ASetYM-zY3AFIkP2Hqu1CiSlktBpxDTVW3hPfy1IgLZRN3jLY9K8WlSvaU9tWiJHfMkQZ2s/s1600/c15.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="802" data-original-width="1424" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqb714dnLd4qKZw6Zif5ovZyOxLxM8dm_hqrvPB1gDHmHZSConeGdvRB1R27PIXNoVHTl8ASetYM-zY3AFIkP2Hqu1CiSlktBpxDTVW3hPfy1IgLZRN3jLY9K8WlSvaU9tWiJHfMkQZ2s/s640/c15.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Different wavelets and quantizer settings can be compared quickly in this software sandbox.</td></tr>
</tbody></table>
That's enough evidence for me that wavelet compression is a completely acceptable trade-off for opening up the possibility of sinking to a normal 1TB SSD instead of an absurd amount of RAM. A very fast RAM buffer is still needed to smooth things out, but it can be limited in size to just as many frames as are needed to ride out pipeline transients. Now, with the source and sink constraints defined, what the hell kind of hardware sits in the middle?</div>
<h4 style="text-align: left;">
The Pipe</h4>
<div>
There was never any doubt that the entrance to this pipeline had to be an FPGA. Nothing else can deal with 64 LVDS channels. But instead of just repackaging the data for PCIe and passing it along to some poor single board computer to deal with, I'm now asking the FPGA to do everything: read in the data, perform the wavelet compression, and write it out to an SSD. This will ultimately be smaller and cheaper, since there's no need for a host computer, but it means a much fancier FPGA.</div>
<div>
<br /></div>
<div>
I'm starting from scratch here, so all of this is just an educated guess, but I think a viable solution lies somewhere in the spectrum of <a href="https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html#productTable">Xilinx Zynq Ultrascale+</a> devices. They are FPGA hardware bolted to ARM cores in a single chip. Based on the source and sink requirements I can narrow down further to something between the ZU4 and ZU7. (Below the ZU4 doesn't have the necessary transceivers for PCIe Gen3 x4 to the SSD, and above the ZU7 is prohibitively expensive.) Within each ZU number, there are also three categories: CG has no extra hardware, EG has a GPU, and EV has a GPU and h.264/h.265 codec.</div>
<div>
<br /></div>
<div>
In the interest of keeping development cost down, I'm starting with the bottom of this window, the ZU4CG. The GPU and video codec might be useful down the road for 30fps previews or making proxies, but they're too slow to be part of the main pipeline. Since they're fairly sideways-compatible, I think it's reasonable to start small and move up the line if necessary.</div>
<div>
<br /></div>
<div>
I really want to avoid laying out a board for the bare chip, its RAM, and its other local power supplies and accessories. The <a href="http://zedboard.org/product/ultrazed-ev">UltraZed-EV</a> <i>almost</i> works, but it doesn't break out enough of the available LVDS pins. It's also only available with the ZU7EV, the very top of my window. The <a href="https://shop.trenz-electronic.de/de/Produkte/Trenz-Electronic/TE08XX-Zynq-UltraScale/">TE08xx Series</a> of boards from Trenz Electronic is perfect, though, covering a wider range of the parts and breaking out enough IO. I picked up the <a href="https://shop.trenz-electronic.de/de/TE0803-02-04CG-1EA-MPSoC-Modul-mit-Xilinx-Zynq-UltraScale-ZU4CG-1E-2-GByte-DDR4-5-2-x-7-6-cm">ZU4CG version</a> for less than the cost of just the <a href="https://www.digikey.com/product-detail/en/xilinx-inc/XCZU4CG-1SFVC784I/122-2262-ND/7034579">ZU4CG on Digi-Key</a>.</div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKCD6QeOUGwqXsHnPUSHQa0MPnbgIlHJU6fFBwnxZ3Cdhf135O6uX4VvSFMvTtqBTxjbcRu5HJwCYKSgDG6vAwgv_ypv_nydfl3GwXOoejAnDdo5FPCkafoAUyB-WEcmqv0ImVazhATNU/s1600/c18.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="960" data-original-width="1600" height="382" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKCD6QeOUGwqXsHnPUSHQa0MPnbgIlHJU6fFBwnxZ3Cdhf135O6uX4VvSFMvTtqBTxjbcRu5HJwCYKSgDG6vAwgv_ypv_nydfl3GwXOoejAnDdo5FPCkafoAUyB-WEcmqv0ImVazhATNU/s640/c18.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Credit card-sized TE0803 board with the ZU4CG and 2GB of RAM. Not counting the FPGA, the processing power is actually good deal less than what's on a modern smartphone.</td></tr>
</tbody></table>
</div>
<div>
One small detail I really like about the TE0803 is that the RAM is wired up as 64-bit wide. Assuming the memory controller can handle it, that would be over 150Gb/s for DDR4-2400, which dwarfs even the CMV12000's data rate. I <i>think</i> the RAM buffer will wind up on the compressed side of the pipeline, but it's good to know that it has the bandwidth to handle uncompressed sensor data too, if necessary.</div>
<div>
<br /></div>
<div>
Time for a motherboard:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTLjwnOwBJyOYMkOktU47tc5uow2UrCey8sN7mDISVWlxBBdCmdonQXSehxhYjgjj2hDZ6tbilfVmPxAh0KK92q0i8so8yTr0Fyqxk-vEyu7496VEOego-KEqrwfUezwhRvi5C4rJlk5Y/s1600/c07.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="861" data-original-width="1600" height="344" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTLjwnOwBJyOYMkOktU47tc5uow2UrCey8sN7mDISVWlxBBdCmdonQXSehxhYjgjj2hDZ6tbilfVmPxAh0KK92q0i8so8yTr0Fyqxk-vEyu7496VEOego-KEqrwfUezwhRvi5C4rJlk5Y/s640/c07.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The "tall" side has the TE0803 headers, an M.2 connector, USB-C, a microSD slot, power supplies, and an STM32F0 to act as a sort-of power/configuration supervisor. Sensor pins are soldered on this side.</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTk6LjCPyFnFnd8NsinfqvDSsP_v-1J9Z360MrIOl7KZtu-PPPf4SbZnjvFaEpRgIvzKp4npm_mbALS5h_u31nRt_bTtgHr3es0Te-Q5B-BeXjvE-zdQBTfRhwkbfwKmwe1JhGUiO-mgE/s1600/c06.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="932" data-original-width="1600" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTk6LjCPyFnFnd8NsinfqvDSsP_v-1J9Z360MrIOl7KZtu-PPPf4SbZnjvFaEpRgIvzKp4npm_mbALS5h_u31nRt_bTtgHr3es0Te-Q5B-BeXjvE-zdQBTfRhwkbfwKmwe1JhGUiO-mgE/s640/c06.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The "short" side has just the sensor and some straggler passives that are under 1mm tall.</td></tr>
</tbody></table>
<div>
Aside from the power supplies, this board is really just a breakout for the TE0803, and the placement of everything is driven by where the LVDS- and PCIe-capable pins are. Everything is a differential pair, pretty much. There are a bunch of different target impedances: 100Ω for LVDS, 85Ω for PCIe Gen3, 90Ω for USB. I was happy to find that <a href="https://jlcpcb.com/">JLCPCB</a> offers a standard 6-layer controlled-impedance stackup. They even have their own <a href="https://jlcpcb.com/client/index.html#/impedanceCalculation">online calculator</a>. I probably still fucked up somehow, but hopefully at least some of it is right so I can start prototyping the software.</div>
<div>
<br /></div>
<div>
Software? Hardware? What do you call FPGA logic? There are a bunch of somewhat independent tasks to deal with on the chip. At the input side, the pixel data needs to be synchronized using training data to deal with the massive 4.8ns inter-channel skew. The FPGA inputs have a built-in delay tap, but it maxes out at 1.25ns. You can, in theory, cascade these with the adjacent unused output delays, to reach 2.5ns. That's obviously not enough to directly cancel the skew, but it <i>is</i> enough to reach the next 300MHz clock edge. So, possibly some combination of cascaded hardware delays and intentional bit slipping can cover the full range. It's going to be a nightmare.</div>
<div>
<br /></div>
<div>
The output side might be even worse. Just look at the number of differential pairs going in to the TE0803 headers vs. the number coming out. That's the ratio of how much tighter the timing tolerance is on the PCIe outputs. The edge of one bit won't hit the M.2 connector until a couple more have already left the FPGA. In this case, I have taken the effort to length match the pairs themselves. I won't know how close I am until I can do a loopback test.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0eng7sfvk8Kf_2snSIDcpuA2f3wphR6qGoxGZyeFDMLL51bLAxpXBllXyBK2WtS-HKAMDd58YB-V8mK5OOHrTfIBL-0dg7mV2vXFBleKU9Z_tUvLyCpvAefaG9RQXdkEsHAZnjdAUlks/s1600/c16.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1028" data-original-width="1600" height="410" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0eng7sfvk8Kf_2snSIDcpuA2f3wphR6qGoxGZyeFDMLL51bLAxpXBllXyBK2WtS-HKAMDd58YB-V8mK5OOHrTfIBL-0dg7mV2vXFBleKU9Z_tUvLyCpvAefaG9RQXdkEsHAZnjdAUlks/s640/c16.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Length matching the PCIe differential pairs to make up for the left turns and TE0803 routing.</td></tr>
</tbody></table>
<div style="text-align: left;">
Even assuming the routing is okay, there's the problem of NVMe. NVMe is an open specification for what lives on top of the PCIe PHY to control communication with the SSD. It's built in to Linux, including versions that can run on the ZU4CG's ARM cores. But that puts the operating system in the pipeline, which sounds like a disaster. I haven't seen any examples of that running at anywhere near 1GB/s. I think hardware-accelerated NVMe might work, but as far as I can tell there are no license-free NVMe cores in existence. I don't have a solution to this problem yet, but I will happily sink hours into anything that prevents me from having to deal with IP vendors.<br />
<br />
Sitting right in the middle, between these input and output constraints, is the complete mystery that is the wavelet core. This has to be done in hardware. The ARM cores and even the GPU are just not fast enough, and even if they were, accessing intermediate results would quickly eat the RAM bus. The math operations involved are so compact, though, that it seems natural to implement them in tiny logic/memory cores and then put as many of them in parallel as possible.<br />
<br />
The wavelet cores are the most interesting part of this pipeline and require a separate post to cover in enough detail to be meaningful. I have a ton of references on the theory and a little bit of concept for how to turn it into lightweight hardware. As it stands, I know only enough to have some confidence that it will fit on the ZCU4CG, in terms of both logic elements and distributed memory for storing intermediate results. (The memory requirement is much less than a full frame, since the wavelets only look ahead/behind a few pixels at a time.) But there is an immense amount of implementation detail to fill in, and I hope to make a small dent in that while these boards are in flight.<br />
<br />
To summarize, I still have no clue if, how, or when any of this will work. My philosophy on this project is to send the pixels as fast as they want to go and try to remove anything that gets in the way. It's not really a plan - more of a series of challenges.</div>
</div>
</div>
</div>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com5tag:blogger.com,1999:blog-8200098102909041178.post-29378777427546067062019-05-27T23:26:00.002-04:002019-10-05T20:57:18.195-04:00KSP: Laythe Colony Part 3, The Colony ShipsJool Launch Window #2 is all about getting as many Kerbals in transit to Laythe as possible, and that means building a fleet of colony ships. This was actually the first ship designed for this mission, but I only built one as a proof-of-concept before committing to the <a href="http://scolton.blogspot.com/2018/11/ksp-laythe-colony-part-2-robotic-fleet.html">Robotic Fleet</a> for Launch Window #1. Those habitats, rovers, and relays will arrive first to pave the way for the colony ships.<br />
<br />
The colony ships are built in orbit, with each part launched separately on the same <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0HF2uYfyZFqsWJ72y3RzPB4hek8Ykmj5evXv4J-b6TLbrKcSPAFqAIW86br6wp7_mBmHGVTBQ8W0XgBy0K3mxXImlCNhQ5nl4Gb5Lyq3dL3UPwWyA9H4_KeM4IUlTDZcwUbIfYCiXRGE/s1600/lc25.png">heavy-lift boosters</a> that sent up the Robotic Fleet. The core of each ship, around which the rest of the ship is built, is a <span style="color: yellow;">Passenger Module</span>:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXCPG84QM_KOMLMJJ5U-chM8o7DNxZ5cPrUuvo7XWa570JewbsnHzIRLj86D1M6XvahhfoLuPfaG-s-756ajkM1o4czMQoe7l6IRvSclE3GSncs-JC1ekdb51w4nP-BIe6FHrS87KwsHE/s1600/lc58.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXCPG84QM_KOMLMJJ5U-chM8o7DNxZ5cPrUuvo7XWa570JewbsnHzIRLj86D1M6XvahhfoLuPfaG-s-756ajkM1o4czMQoe7l6IRvSclE3GSncs-JC1ekdb51w4nP-BIe6FHrS87KwsHE/s640/lc58.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Passenger Module</td></tr>
</tbody></table>
<div style="text-align: left;">
The Passenger Module has room for 18 Kerbals (half the crew of each ship), with two main living compartments on each end, a central stack of general purpose seating, and two observation domes. It's meant to be the "comfortable" portion of the ship, to make the multi-year journey more bearable than would be possible in a lander cockpit. Not that Kerbals really care.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
One of the main quality of life considerations for the colony ships is the ability to spin to generate artificial gravity in some of the living quarters. For this reason, the rest of the ship is built along an axis passing through the center of the passenger module. Forward, the next part is the <span style="color: yellow;">Docking Module</span>:</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQhb_SPAY-escXHvvrlUbifR78AGtPCvkOck8WmTazmGnq7hC-vb4UqtnP3PCvLMOzTmXAV_MMaMhqtfianlD41s9YOcX0qtqYKcRka-Rwt83ZDA4Mo43n4H-6U0TfoQ2Do6cQ7TYoKWg/s1600/lc71.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQhb_SPAY-escXHvvrlUbifR78AGtPCvkOck8WmTazmGnq7hC-vb4UqtnP3PCvLMOzTmXAV_MMaMhqtfianlD41s9YOcX0qtqYKcRka-Rwt83ZDA4Mo43n4H-6U0TfoQ2Do6cQ7TYoKWg/s640/lc71.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Docking Module</td></tr>
</tbody></table>
<div style="text-align: left;">
While it has space for another six Kerbals, the Docking Module is more of a working space than a living space. Since it's on the central axis, there's no artificial gravity. But it has a large science lab and common area for the crew. Most importantly, it serves as the docking interface for the <span style="color: yellow;">Space Planes</span>, which shuttle crew to and from the colony ships.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLgmwxEh5U0LXuKVQHnbEoCM6SmAJ8EiLtNRlxtbny8MW90noG_P1VBr1W0177QcHBJX-F7cVwfZZ7s0_qaYSkfVrnWP7QYp2_BQZmTJ-OSAzAackCxgG7Ahw57liGopndVKxexJjDRug/s1600/lc23.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLgmwxEh5U0LXuKVQHnbEoCM6SmAJ8EiLtNRlxtbny8MW90noG_P1VBr1W0177QcHBJX-F7cVwfZZ7s0_qaYSkfVrnWP7QYp2_BQZmTJ-OSAzAackCxgG7Ahw57liGopndVKxexJjDRug/s640/lc23.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Space Planes</td></tr>
</tbody></table>
<div style="text-align: left;">
The Space Planes are really the key to this entire mission, providing a way to get hundreds of Kerbals down to Laythe without having to <a href="http://scolton.blogspot.com/2014/03/ksp-mission-to-laythe-and-back.html">exactly target flat landing sites from orbit</a>. I tweaked and tested the design to the point where getting to orbit, docking with a colony ship, and returning to Kerbin for a runway landing was utterly routine. Each colony ship required four Space Plane round-trips and two one-way trips to fully crew. The two one-ways go with the ship to Laythe, where they will be used to ferry Kerbals down to the surface.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The back-to-back docking configuration for the Space Planes minimizes the moment of inertia along the spin axis. The two planes have to be exactly symmetric, so each interfaces with two medium-size docking ports for alignment. It is possible, with careful flying, to get both ports to engage at the same time. In addition to enforcing symmetry, this makes the final structure much more rigid. Finding parts that are exactly the right spacing on both sides to make this possible was the trickiest part of the design.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I could write an entire post about the Space Plane design, but I think I'll just post some pictures and videos of it <b>kicking ass</b> instead:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdMm1YUKpk3sL68KTFWYfU7hnnh-yChS1G3jf1DfcrzNslZOCvKs0PYULcYrnByAL8L5VIbLcDSO38B9uUMpH0LvKtCpulmx7mMkASwSDoL4D97MGqZHV7cKlkfzxdnOI3eodNvztP3tE/s1600/lc98.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdMm1YUKpk3sL68KTFWYfU7hnnh-yChS1G3jf1DfcrzNslZOCvKs0PYULcYrnByAL8L5VIbLcDSO38B9uUMpH0LvKtCpulmx7mMkASwSDoL4D97MGqZHV7cKlkfzxdnOI3eodNvztP3tE/s640/lc98.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiKQsO7hflybcc-EcUQxujnb0PLxT4GVxERmscAV-I-N-fElQ9qjSz_8nYGCOUco2xDAT9csaWlgrZW5kHn_8-TXUJ6SejnCYypselGvWodLVZsEWufu_c3QEH5N_0fDtbu3PToEviphE/s1600/ld25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiKQsO7hflybcc-EcUQxujnb0PLxT4GVxERmscAV-I-N-fElQ9qjSz_8nYGCOUco2xDAT9csaWlgrZW5kHn_8-TXUJ6SejnCYypselGvWodLVZsEWufu_c3QEH5N_0fDtbu3PToEviphE/s640/ld25.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/TF1-GfccDro" width="640"></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0U14FefxFo_9iiorcsCOyOwez5ZyKSJRGiehcXqC9JdpqfDawWmrfCf1oHKJ6-6uyWUze4jmZPUk69Q6jymJwX2FBZg8B5mFtu0zjPBjlzoV_eM0gbWYHIOjFPhS-yKiY_X_j992Y0d0/s1600/lc66.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0U14FefxFo_9iiorcsCOyOwez5ZyKSJRGiehcXqC9JdpqfDawWmrfCf1oHKJ6-6uyWUze4jmZPUk69Q6jymJwX2FBZg8B5mFtu0zjPBjlzoV_eM0gbWYHIOjFPhS-yKiY_X_j992Y0d0/s640/lc66.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWYkhfYrr4Bmu8o5zDA-4JxxQhLqUGVeApJUOi5xxvRkI8MEaZt-XG5iUtcVWs20CBsK3qwmZFvzF5IVJEqdg2OAXuTDmSqFhWt6UkgKeUblX0W84MYiof0khZTPm-5lmWpkkGAKYXoI0/s1600/ld03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWYkhfYrr4Bmu8o5zDA-4JxxQhLqUGVeApJUOi5xxvRkI8MEaZt-XG5iUtcVWs20CBsK3qwmZFvzF5IVJEqdg2OAXuTDmSqFhWt6UkgKeUblX0W84MYiof0khZTPm-5lmWpkkGAKYXoI0/s640/ld03.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieabK_0HJiXtXs6m2gG7e76OQlx2F2k0FRp_kuBamGRnPJZrPukVbD7CIBGIPpxFgzCh9NoHxuATUP4cHmAKeJ67iyWO7i0rTq2UdbnPkt1PISvd3ULk2yVaK3IndDsO8OGT6uCyMUdzk/s1600/ld04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieabK_0HJiXtXs6m2gG7e76OQlx2F2k0FRp_kuBamGRnPJZrPukVbD7CIBGIPpxFgzCh9NoHxuATUP4cHmAKeJ67iyWO7i0rTq2UdbnPkt1PISvd3ULk2yVaK3IndDsO8OGT6uCyMUdzk/s640/ld04.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The last picture has a story that goes with it: For some reason, after dozens of clean flights, I botched a take-off and slammed back into the KSC runway with the gear still down, breaking off both wings, the outer engines and fuel tanks, the vertical stabilizers, and all but the two inner horizontal control surfaces. The fraction of a plane that was left was somehow still able to gain altitude, do a wide 180º turn, and make a water landing just off shore.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Anyway, back to the colony ships. Behind the Passenger Module is a truss structure I just call <span style="color: yellow;">The Node</span>:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPcJKj2AvhgZmedj2P7EXa_A-yAmkue4JJysl_NnxuUGbvCqOkQAkQEvPAnoXp7cP4jBIC1-2-1U81nos8kYqXScWXG-avLBHnGqJ6x-2ZqtXGuL7QBMvNOpaM1fEPF3J9uC-uN9G5Ugs/s1600/lc20.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPcJKj2AvhgZmedj2P7EXa_A-yAmkue4JJysl_NnxuUGbvCqOkQAkQEvPAnoXp7cP4jBIC1-2-1U81nos8kYqXScWXG-avLBHnGqJ6x-2ZqtXGuL7QBMvNOpaM1fEPF3J9uC-uN9G5Ugs/s640/lc20.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Node</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
This is the lightest and simplest part of the colony ship, primarily serving as a connector between the crew stack and the propulsion. It also carries the large solar panels, some battery storage, extra reaction control systems, and side ports for docking other modules, such as for refueling. Altogether a small but important building block.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
To push all this, three <span style="color: yellow;">Propulsion Modules</span> are launched separately and docked to the back of the Node. These are the full four-engine versions of the <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmET2hwOfsPHswgNSmmru0DsAKFfhZpfk_bfHC28nTUjj41WH0oAOZjZe7QYbsPbRBth5vtD_q-VNvHXy_Eyv310UBGOcb8GdYjZzuFHlFZha3YeYxO17BP-VYVYQYN5V4ZFBMqjEqmG8/s1600/lc40.png">propulsion modules used for the Robotic Fleet</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIrt9LEaEPQ7IreEwP66n_rlNqgMSvNM9CmXbz1GdoGu7mHVNpF6YV90qgVRXC49gKiR5I2aT0RsWk2pAS3kjf9BRbD_PaCo8Sk8YNATcrFG7ldMLZuUcp9cnrrJDOBnnzqaui_fF64Ec/s1600/lc61.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIrt9LEaEPQ7IreEwP66n_rlNqgMSvNM9CmXbz1GdoGu7mHVNpF6YV90qgVRXC49gKiR5I2aT0RsWk2pAS3kjf9BRbD_PaCo8Sk8YNATcrFG7ldMLZuUcp9cnrrJDOBnnzqaui_fF64Ec/s640/lc61.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Propulsion Modules</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
With 12 engines in total, the colony ships actually have a higher thrust-to-weight ratio than the robotic landers. The entire colony ship, including the two Space Planes, comes in at just under <b>300 tons</b> and has a fully-fueled Delta-V of about <b>4400m/s</b>, which should be enough to get to Laythe orbit with just a tiny bit of help from gravity assists off Tylo (or Laythe itself).</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The process of building a single colony ship takes 13 separate launches: six for assembly, six crew shuttles (including two permanent ones), and one refueling run. (While the propulsion modules get to orbit fully-fueled, the total Delta-V counts on topping off the two Space Planes.) It's without a doubt the most ambitious in-orbit construction project I've attempted in KSP.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/rycBSWTirDU" width="640"></iframe></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Oh, and I built 10 of these for Launch Window #2. That's about <b>3,000 tons of hardware, including 20 Space Planes and 360 Kerbals</b>, on the way to Laythe.</div>
<h4 style="clear: both; text-align: left;">
196 Days Remain</h4>
<div class="separator" style="clear: both; text-align: left;">
I had intended for the second launch window to be the last ships out, but my arbitrary deadline of Year 3, Day 0 for the destruction of Kerbin leaves some time to send up a few more. They'll have to wait for the third launch window, possibly in the relative safety of a Minmus orbit, but I can think of a few extra pieces of hardware that would be useful to the colony.</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com0tag:blogger.com,1999:blog-8200098102909041178.post-80758865438078300752019-03-31T12:03:00.000-04:002019-03-31T12:04:53.626-04:00TinyCross: Chassis Build<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqTjkPqS-UnyOMW4A-N_Tz75VgSmBe34Xu0CXs1jGy3vF1tdxccVDYgUA1-KUjyMvqSe8GatsRTYmDBjxCHzNAqHArPI2L1BehpVYr0Wh2mGkPC37enyqvrpgVACvXDv6GvOAOjTsIfuM/s1600/tc55.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="487" data-original-width="1600" height="194" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqTjkPqS-UnyOMW4A-N_Tz75VgSmBe34Xu0CXs1jGy3vF1tdxccVDYgUA1-KUjyMvqSe8GatsRTYmDBjxCHzNAqHArPI2L1BehpVYr0Wh2mGkPC37enyqvrpgVACvXDv6GvOAOjTsIfuM/s640/tc55.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Just hit print...</td></tr>
</tbody></table>
<div style="text-align: left;">
This winter build season is coming to a close: almost all of the mechanical work for <a href="http://scolton.blogspot.com/2018/08/tinycross-ultralight-electric-crosskart.html">TinyCross</a> is done! Here's a recap how the rolling chassis came together:<br />
<br /></div>
<div style="text-align: left;">
The frame and suspension is mostly a kit of aluminum plates and 80/20 extrusion, with almost no post-machining required, so it went together very quickly. After the main box frame assembly and seat mounting, I started with a test build of a single corner of A-arm geometry to make sure I hadn't missed any clearance issues. I also wanted to get a first impression of the stiffness in real life, since that's probably the biggest risk of this new design.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEighStyanaMJDLr8KvkMwf_Ad0MgOaVq1qD5qu1y_u6NSqrZfRNkNFeJKeatUngI25U2aVZdKruu9XwdgIn6DMtg85vhdwr12HOifeXuhvka9UzEzhcVoQiEyp6zdvIsvLtlHADM4WwFX0/s1600/tc24.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEighStyanaMJDLr8KvkMwf_Ad0MgOaVq1qD5qu1y_u6NSqrZfRNkNFeJKeatUngI25U2aVZdKruu9XwdgIn6DMtg85vhdwr12HOifeXuhvka9UzEzhcVoQiEyp6zdvIsvLtlHADM4WwFX0/s640/tc24.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
I had no trouble at all with the front-right corner. Everything fit together as planned and the stiffness felt adequate, largely thanks to the zero-slop <a href="https://www.summitracing.com/parts/qa1-cmr4ts/overview/">QA1 ball joints</a>. I was actually a little surprised at how well-behaved it felt. Once all six ball joints and the <a href="https://www.amazon.com/DNM-Mountain-Shock-Lockout-190mm/dp/B00FLTZ47O/ref=asc_df_B00FLTZ47O/?tag=hyprod-20&linkCode=df0&hvadid=309768114180&hvpos=1o2&hvnetw=g&hvrand=6007427895477526084&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9002012&hvtargid=pla-569284672209&psc=1">air shock</a> pins were tightened down, it really did have only the degrees of freedom it should have: one for steering and one for suspension travel. There's no rattling or play at all. With high confidence from the test corner, I went into production line mode for the other three.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE7XuYAhB6uiQq60pb3EeOkXRYHm9zlQDulIXvVRVxhdj-a-126Vaxikmal0K6FYYF1oDZ1xU2ompmKLEObUdw1LXRdRXgEJOAL9FflcgMipxceHsaT7AeWC_sNcvDpH772tinEN_9Lmk/s1600/tc26.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE7XuYAhB6uiQq60pb3EeOkXRYHm9zlQDulIXvVRVxhdj-a-126Vaxikmal0K6FYYF1oDZ1xU2ompmKLEObUdw1LXRdRXgEJOAL9FflcgMipxceHsaT7AeWC_sNcvDpH772tinEN_9Lmk/s640/tc26.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">PTFE-lined QA-1 1/4-28 rod ends (<a href="https://www.summitracing.com/parts/qa1-cmr4ts/overview/">CMR4TS</a>) are the real stars of this build. It would not be possible with McMaster's selection of ball joints, which are either cheap and overly loose or expensive and overly tight.</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA6eFRfR9-RuER57jnWSKUv-AbmUZiqDVNi87gr3GtkkDovOphTBn8FM88WFLb_kDKQEwinZwRkWMXcC9z26hLpX4cj_f7aSGstd-V1KCpnRamYN-KkPS8aqovPnEuyNe2xfJDxJNhnSk/s1600/tc31.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA6eFRfR9-RuER57jnWSKUv-AbmUZiqDVNi87gr3GtkkDovOphTBn8FM88WFLb_kDKQEwinZwRkWMXcC9z26hLpX4cj_f7aSGstd-V1KCpnRamYN-KkPS8aqovPnEuyNe2xfJDxJNhnSk/s640/tc31.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">I say there was almost no post-machining, but just tapping all the 80/20 ends was a whole day of work.</td></tr>
</tbody></table>
<div style="text-align: center;">
<div style="text-align: left;">
About here is where the perfect build ended, though, because when I went to attach the front-left corner, I discovered that there was a slight interference between the A-arms and the air shock valve stems. The parts I designed, all 2D plates, are 100% symmetric, so I didn't bother to model the other corners. But the shocks themselves are not symmetric, so it wasn't exactly correct to assume that things would fit together the same in the mirrored configuration. Since the interference was minimal, I debated cutting notches in the A-arms for the valve stems. But I was able to find a more satisfying solution.</div>
</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMFNJl6hNWWmYw4C-a_j6eR2IOlTVYgoqCh1_8mdfQ-xZoI5lRLkeNyzT3R1co62a_YHYCKJz76N_csJhtIZtAf5A7vt9U-ymGgFr4iryvAzCjCd7RPOYklx2SeAoRy0JRWF5sITvqI6s/s1600/tc25.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMFNJl6hNWWmYw4C-a_j6eR2IOlTVYgoqCh1_8mdfQ-xZoI5lRLkeNyzT3R1co62a_YHYCKJz76N_csJhtIZtAf5A7vt9U-ymGgFr4iryvAzCjCd7RPOYklx2SeAoRy0JRWF5sITvqI6s/s640/tc25.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Can you spot it?</td></tr>
</tbody></table>
Not modeling the mirrored parts was a semi-legitimate time-saving strategy, but not modeling the rear corners was just laziness. And of course there was a major interference there: the brake calipers would not have cleared the corners of the frame. (In the front, it's no problem since there is extra space for steering travel.) It was nothing that couldn't be solved with a hacksaw and some improvisation, though. I actually like the final outcome better than the original design...<br />
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQwfmWIlg3K0Ul_5vFVmZV1ig-RGKEE5Uf50PoFX1yeaFp_nBpUSEG674ro-ojFlYHem3xQQOF320bZypYHKNowEUVDua2yJP4WJjo0Ddy2pk44mmpI3b5lLF68ueHj8HP168QMGoj8S8/s1600/tc36.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQwfmWIlg3K0Ul_5vFVmZV1ig-RGKEE5Uf50PoFX1yeaFp_nBpUSEG674ro-ojFlYHem3xQQOF320bZypYHKNowEUVDua2yJP4WJjo0Ddy2pk44mmpI3b5lLF68ueHj8HP168QMGoj8S8/s640/tc36.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">...he justifies, in post.</td></tr>
</tbody></table>
<div style="text-align: left;">
Minor issues aside, I am pleased with how the chassis turned out. It's much stiffer than <a href="http://scolton.blogspot.com/p/cap-kart.html#tinykart">tinyKart</a>, thanks to a slight excursion into the third dimension, but still very light. And I went from 50% to 99% confidence on the suspension design after getting hands on the assembled corners.</div>
</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHKJoXeMa_IIDDFUDAM8QFB5P2Pc_fbmairJt_Mjs60DUrGJ0tV3KUCcfOYYk5CX92v4AAcqAxxBFcDI_Lgu2PGc2InX3XcAbb1LOH2XG2AT4IiDghZGBJMwz-GN_tUMdH_aoS3-lP_gE/s1600/tc33.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1200" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHKJoXeMa_IIDDFUDAM8QFB5P2Pc_fbmairJt_Mjs60DUrGJ0tV3KUCcfOYYk5CX92v4AAcqAxxBFcDI_Lgu2PGc2InX3XcAbb1LOH2XG2AT4IiDghZGBJMwz-GN_tUMdH_aoS3-lP_gE/s640/tc33.jpg" width="480" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">So far, so light.</td></tr>
</tbody></table>
<div style="text-align: left;">
Most of the machining for this build was for turned parts within the four drive modules. The spindle shafts for the wheels were made from 7071 aluminum and support the wheel bearings (6902-2RS). Unlike on tinyKart, the shafts are doubly-supported within a box structure built around the wheel, which should be much more impact tolerant. The large drive pulleys got some weight reduction and a custom bolt pattern to interface with the wheel hubs.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxvAHBZLi3UKRLT8J6rTSA5cfMsd2UYBQkd2K11XXIvU1m51jgfVK-_HpN9FsYFN1NCZ0-kEQvWcjz7mn7PvGg0I7eZk0EfzVwaC5WeXWC54tdEvFewVAZw5YcaeVhrH5mEd_ZFKYVC0/s1600/tc39.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="586" data-original-width="1600" height="234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxvAHBZLi3UKRLT8J6rTSA5cfMsd2UYBQkd2K11XXIvU1m51jgfVK-_HpN9FsYFN1NCZ0-kEQvWcjz7mn7PvGg0I7eZk0EfzVwaC5WeXWC54tdEvFewVAZw5YcaeVhrH5mEd_ZFKYVC0/s640/tc39.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
My favorite bit of packaging is the brake caliper occupying the volume inside the belt loop, with the brake disk flush against one side of the wheel pulley. Torque is sourced and sunk from the same side of the wheel - in fact from the same metal plate.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHlv06ht8dWKMU1OJaMn7kL0ihYD7FNoiw5jppYan25VJz3hm9HdQFQe0MpLTD5hyScbgnubMXq94Y4tKE1UQJJxsTPTobp91hEmZejWKOU-YN9mjDNoDfXNIdlCgb8_rF27adhopUUDY/s1600/tc47.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1290" data-original-width="1600" height="514" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHlv06ht8dWKMU1OJaMn7kL0ihYD7FNoiw5jppYan25VJz3hm9HdQFQe0MpLTD5hyScbgnubMXq94Y4tKE1UQJJxsTPTobp91hEmZejWKOU-YN9mjDNoDfXNIdlCgb8_rF27adhopUUDY/s640/tc47.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
The motor shaft and motor pulley also required some custom machining. This was a weak link on tinyKart: The original design used set screws on shaft flats, but it was prone to loosening over time (or, in one case, completely shearing off the 10mm shaft at the flat). After switching to keyed shafts (via <a href="https://alienpowersystem.com/shop/brushless-motors/aps-6374hev-outrunner-brushless-motor-170kv-3300w/">Alien Power Systems</a> motors), those problems mostly went away. But there was still axial play, and the torque was still being transmitted through a 3mm key into an aluminum keyway.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
For TinyCross, I wanted to have a clamping and keyed shaft adapter so the torque would be primarily transmitted through friction, with the key as back-up. There's not a lot of room to work within the 15-tooth drive pulley, so that just gets bored out as much as possible and then pressed like hell, with retaining compound, onto a 7071 adapter. This adapter then gets the 10mm bore with a 3mm keyway. But it also gets slotted, turning it into a clamp. Finally, an off-the-shelf 0.75in aluminum clamping collar tightens the whole assembly down onto the motor shaft, with the key in place.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJu4cxOrl_lHJJmUsiZe1pYUSPuuOfXgxvgNW2FTnuLJguAWx49p419CoP_8j5dTIAu7Eo9fq7MTD4NhJClI1uY6fXKrKsVg-vRKhuzURwjemwldluqfBNypdCtt0eK77ETMcTRimE5Is/s1600/tc56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="874" data-original-width="1600" height="347" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJu4cxOrl_lHJJmUsiZe1pYUSPuuOfXgxvgNW2FTnuLJguAWx49p419CoP_8j5dTIAu7Eo9fq7MTD4NhJClI1uY6fXKrKsVg-vRKhuzURwjemwldluqfBNypdCtt0eK77ETMcTRimE5Is/s640/tc56.png" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
Additionally, the outboard side of the shaft interfaces with another bearing, for double support, and has a pocket for a shaft rotation sense magnet, to be picked up by a rotary encoder IC.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilJISp60tT798cW6-aGZnZHR8PoSP5MKn86tuNp9h_P99HNafLkd93JbP3iP7q_95gUmp7sh3bWQCCELo4Y5TSBpK8LplH8jk2IQwVzOIAfwiBHKRBajjda_ncLvM-RlF3cA6qMRTZY_Q/s1600/tc53.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilJISp60tT798cW6-aGZnZHR8PoSP5MKn86tuNp9h_P99HNafLkd93JbP3iP7q_95gUmp7sh3bWQCCELo4Y5TSBpK8LplH8jk2IQwVzOIAfwiBHKRBajjda_ncLvM-RlF3cA6qMRTZY_Q/s640/tc53.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Not messing around.</td></tr>
</tbody></table>
<div style="text-align: left;">
For brakes, I opted for the same disks, calipers, cables, and levers as on tinyKart. I briefly debated going hydraulic, but the plumbing for four wheel disk brakes seemed like an unnecessary nightmare. tinyKart never had a problem with braking torque; it could easily lock up both front wheels. It just had so little weight on the front wheels that braking and steering were often mutually exclusive activities. With four wheel disk brakes, TinyCross should be much more controllable under braking. The <a href="https://www.amazon.com/TerraTrike-Dual-Control-Brake-Lever/dp/B001FYEOIE">TerraTrike dual pull levers</a> are key to making this work: they have a fulcrum between the lever and two cable ends that ensures both cables get pulled with equal force. I have one such lever for the two front discs and one for the rears.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4kN9LtwWdPH5lQSl4HCXReI1YzhVM7rtIFTSs8IO3NojQPK6hAcFzQwfMWSg42z4ZI0chHA0rD2yuvXrsDs8nYLuvYVcivEy0BuaE5UKL42_fNClpHDQM87F14M8OROULbWn-ESLmoa8/s1600/tc52.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4kN9LtwWdPH5lQSl4HCXReI1YzhVM7rtIFTSs8IO3NojQPK6hAcFzQwfMWSg42z4ZI0chHA0rD2yuvXrsDs8nYLuvYVcivEy0BuaE5UKL42_fNClpHDQM87F14M8OROULbWn-ESLmoa8/s640/tc52.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Independent rear brake lever...what could go wrong?</td></tr>
</tbody></table>
<div style="text-align: left;">
The last piece of the mechanical puzzle is the steering. At each wheel, there's a steering arm that terminates in a place to mount yet another ball joint, using a T-nut. This is driven by a link comprising a threaded rod with an aluminum stiffener, another trick carried over from tinyKart. The aluminum stiffener is compressed and the threaded rod is stretched by two nuts, creating a link that's stiffer than either part by itself.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj24K4XOrY3EPl0cMNQ7lFITdCF0HcmGOW5NERqZqVWzJdHmmzT1ydJGoeqfJ8WNknNGKyRbn0lx_8tShT_jvkZmxZk9qFEtbZ3G5cWX2T1UgNH3NKQHNSLFn4cGADnRwEbQci1l1HfCjc/s1600/tc48.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1561" data-original-width="1600" height="624" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj24K4XOrY3EPl0cMNQ7lFITdCF0HcmGOW5NERqZqVWzJdHmmzT1ydJGoeqfJ8WNknNGKyRbn0lx_8tShT_jvkZmxZk9qFEtbZ3G5cWX2T1UgNH3NKQHNSLFn4cGADnRwEbQci1l1HfCjc/s640/tc48.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWaEeNBYcxBZC-uMBPHevqhXISkbXvpsRi5QiZkJvfDpMwdP4exhVYN_RcsQbXfdAbMU-VYL0lecwx3hVXk2BpeZxlgYv0SPC2T1xHEPb3mYlHYFiQjqGuv986EYPAbdjzfbcLYOJfeYE/s1600/tc54.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="447" data-original-width="1600" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWaEeNBYcxBZC-uMBPHevqhXISkbXvpsRi5QiZkJvfDpMwdP4exhVYN_RcsQbXfdAbMU-VYL0lecwx3hVXk2BpeZxlgYv0SPC2T1xHEPb3mYlHYFiQjqGuv986EYPAbdjzfbcLYOJfeYE/s640/tc54.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: left;">
For the rear wheels, all that's required are two fixed mounting points for the other end of these rods. Rear toe angle is set by adjustment with the threaded rod.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGu51FBr8F8w36-8VamMqMfjmx291MdmBLCm-emh2Act6MCwnC4A9TagdRThC6ZWGW4wnw2uo_ijSG78MBSifGDeuWnPJlHmwWnRP4ZYOp43RnWMszny7eZxz4EfVyPpCFkXFm6Bu0Jgs/s1600/tc50.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGu51FBr8F8w36-8VamMqMfjmx291MdmBLCm-emh2Act6MCwnC4A9TagdRThC6ZWGW4wnw2uo_ijSG78MBSifGDeuWnPJlHmwWnRP4ZYOp43RnWMszny7eZxz4EfVyPpCFkXFm6Bu0Jgs/s640/tc50.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The toe-setting plate doubles as the TinyCross badge.</td></tr>
</tbody></table>
<div style="text-align: left;">
The front requires an actual steering mechanism. In lieu of a rack-and-pinion, I used a simple four-bar, driven by the steering column through a universal joint buried in the middle of the front suspension support tower. Each link in the four-bar has its own set of thrust bearings and radial bushings, to minimize the extra linkage slop.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDpVEzv2iRVspk6MUZ3XRSVRdyxIXXdSrF813Scqo4NChG188BaoEYnJsL07ZAH7cCBZX1nbda-0iaQQt4iNfzHxno31M-1m96v25umgKzIWcpg_PdVWcPnKonQmq3LIxczL2WFVPfeoI/s1600/tc45.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDpVEzv2iRVspk6MUZ3XRSVRdyxIXXdSrF813Scqo4NChG188BaoEYnJsL07ZAH7cCBZX1nbda-0iaQQt4iNfzHxno31M-1m96v25umgKzIWcpg_PdVWcPnKonQmq3LIxczL2WFVPfeoI/s640/tc45.jpg" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6lnViKA2o-yAv8T6jRbtikc7d3RSlqFPnoIxZW-8Z9yKc0yP7rVgjXUNyftSna27_7jV1ULefQ0ZM12dqHO7eRc97w-tT2g8lrFw2sQ7a7O-HAoYK9PP1m8xdLMQ4VQhzKTQAh8csL7o/s1600/tc46.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="979" data-original-width="1600" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6lnViKA2o-yAv8T6jRbtikc7d3RSlqFPnoIxZW-8Z9yKc0yP7rVgjXUNyftSna27_7jV1ULefQ0ZM12dqHO7eRc97w-tT2g8lrFw2sQ7a7O-HAoYK9PP1m8xdLMQ4VQhzKTQAh8csL7o/s640/tc46.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
This sort of setup works since the steering throw is very short: ±45º of travel is all it needs. Amazingly, it all clears over the full suspension travel and there doesn't seem to be much bump steer. (I shouldn't be amazed, since it works in CAD, but I am anyway.)</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
And just like that, it rolls.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPZmBcN7w4teCtsPqW3ywdwVEZa0-LFKWRgRN5WXlwIk869IY6tfJ_GOBJ0drJRRr3sKZvMyyarDatb2cr-GONZiis7A5j1kZPFet55wcJRFqQa0wsIB-LZUfYmI14JoiBrBvZ51ZAxaM/s1600/tc51.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPZmBcN7w4teCtsPqW3ywdwVEZa0-LFKWRgRN5WXlwIk869IY6tfJ_GOBJ0drJRRr3sKZvMyyarDatb2cr-GONZiis7A5j1kZPFet55wcJRFqQa0wsIB-LZUfYmI14JoiBrBvZ51ZAxaM/s640/tc51.jpg" width="640" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
Unfortunately, although the frame and suspension were on-target, the four power modules are way over weight budget. The wheels themselves are annoyingly heavy. I can't do much about that, but I can probably take some weight out of the surrounding assembly. A lot of the design is driven around the off-the-shelf cast aluminum rims. If I am willing to chop down the rim and re-machine its outer bearing bore, I can probably save a little weight and a lot of width. I can maybe even get it below 34in, which would help with getting through doorways.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
But for now I will shift focus to the electronics. Preliminary bring-up of the motor drives has been uneventful (that's a good thing), but I'll need to actually hook them up to LiPos next, which could always become interesting in fiery ways.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhosdlK0nRKyTuFDanpia_6VG0nrgrd1PskQTe7PsL9kQ20jc8rHNQI6hA0a58AL4Wc4rpyuKCdXtfSHDN6URmGFNClXyOQkIAhIFxHYEI0A5MzaqJVg_06O2qyYa6MkCw27S7lGi8JFHU/s1600/tc55.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhosdlK0nRKyTuFDanpia_6VG0nrgrd1PskQTe7PsL9kQ20jc8rHNQI6hA0a58AL4Wc4rpyuKCdXtfSHDN6URmGFNClXyOQkIAhIFxHYEI0A5MzaqJVg_06O2qyYa6MkCw27S7lGi8JFHU/s640/tc55.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">To be continued...</td></tr>
</tbody></table>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com2tag:blogger.com,1999:blog-8200098102909041178.post-1466879595280471832018-11-29T23:10:00.000-05:002018-11-29T23:23:26.516-05:00KSP: Laythe Colony Part 2, The Robotic Fleet and Launch Window #1In honor of the successful <a href="https://mars.nasa.gov/insight/">Mars InSight landing</a> this week, I thought I'd do a progress report on my long-term KSP mission to get as many Kerbals off Kerbin by Year 2, Day 0 as possible. <a href="http://scolton.blogspot.com/2017/08/ksp-laythe-colony-part-1.html">Part 1</a> sets up the premise and the main strategy. In this Part 2, I throw about 1,000 tons of robotic hardware at Jool during the first available launch window, with hopes that at least some of it winds up in a single spot on the surface of Laythe as the seed for a colony.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiR9DIkhR88jlmSZkFn5bp6ZJyXekNiwi4Twj3ZULsR8vHnzjan93L6QW1lpKA9URa6BEDmgtjsPUpks6WxhNMt6sO2A_K0fOcqIElALCQXFoRHhB76TK-dmGbC0AkYyCgu6Kq1tUks3Pg/s1600/lc35.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="932" data-original-width="1600" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiR9DIkhR88jlmSZkFn5bp6ZJyXekNiwi4Twj3ZULsR8vHnzjan93L6QW1lpKA9URa6BEDmgtjsPUpks6WxhNMt6sO2A_K0fOcqIElALCQXFoRHhB76TK-dmGbC0AkYyCgu6Kq1tUks3Pg/s640/lc35.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The busy 1000m/s on-ramp to Jool Transfer Orbit.</td></tr>
</tbody></table>
<div style="text-align: left;">
<h4>
The Robotic Fleet</h4>
For the first launch window, I decided to send only uncrewed vehicles to feel out the Jool transfer orbit, the details of maneuvering within the Jool system, and the landing procedures at Laythe. The robotic fleet consists of three types of ship: Triple Relay Satellites (RS3), Laythe Rovers (LR1), and Habitats (HAB1). Each one has a different function crucial to settling a remote colony.</div>
<h4 style="text-align: left;">
RS3</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGAejoOhW50xfB_laBGHDCvZOtJEJclZCYKQeX_KHILCy2GkQlu1LB_qoPt8lZqxSzrPPvEzNGrT9Xo3LyMtM4luOI_nQhHIOTgp5Y9vU7L3Ask8p5W7wAsT1vN8690K8su-f9IuuQoY/s1600/lc10.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGAejoOhW50xfB_laBGHDCvZOtJEJclZCYKQeX_KHILCy2GkQlu1LB_qoPt8lZqxSzrPPvEzNGrT9Xo3LyMtM4luOI_nQhHIOTgp5Y9vU7L3Ask8p5W7wAsT1vN8690K8su-f9IuuQoY/s640/lc10.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Does this thing get HBO?</td></tr>
</tbody></table>
<div>
These are the smallest and lightest ships, but critical to this first remote-controlled mission phase. One of the relatively new realism additions to KSP is that uncrewed vehicles need to have a line-of-sight communication path back to Kerbin, or to a ship with a crew, in order to maneuver. To achieve this requires lining up a bunch of relay satellites around Kerbin and at other useful locations in the system. </div>
<div>
<br /></div>
<div>
Each RS3 assembly carries three small relay satellites with their own ion drives. In addition to the ones already parked around Kerbin, two sets of three are on their way out to Kerbin <a href="https://en.wikipedia.org/wiki/Lagrangian_point#L4_and_L5">L4 and L5</a> stations and then eventually other equally spaced points in the orbit. Three more sets are heading out to an intermediate orbit between Kerbin and Jool. And four sets of three are in the fleet heading for Jool, to set up a network around Laythe.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2hOdKD1UWiovwfyp38DNwR89tdp3Nxa3kNh2_PfY6UNRl6G2CKifiX2TeFg1b7Mpp_JBclb3M2psc0psmVdpcb-QsFjFEZ2ayQcToAXwVJteD_e6sJK_Rd5LQcyV6l9eHBdLJBmTQA1o/s1600/lc00.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2hOdKD1UWiovwfyp38DNwR89tdp3Nxa3kNh2_PfY6UNRl6G2CKifiX2TeFg1b7Mpp_JBclb3M2psc0psmVdpcb-QsFjFEZ2ayQcToAXwVJteD_e6sJK_Rd5LQcyV6l9eHBdLJBmTQA1o/s640/lc00.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The start of the mission's comms network.</td></tr>
</tbody></table>
<h4 style="text-align: left;">
LR1</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZWmFzg5Z4huWtRLWFtkb-2jTLMgM-hOXPmovRiX14UNBjnRUbLX6zLqpMmQQw0bSUPFM7XVa1fTtUV76s88rhvugpNAGWSiUkrO2YUVFlqLAA1YaQplTRGy1qYM-OHiDpEa-uOt30jeI/s1600/lc37.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="724" data-original-width="1600" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZWmFzg5Z4huWtRLWFtkb-2jTLMgM-hOXPmovRiX14UNBjnRUbLX6zLqpMmQQw0bSUPFM7XVa1fTtUV76s88rhvugpNAGWSiUkrO2YUVFlqLAA1YaQplTRGy1qYM-OHiDpEa-uOt30jeI/s640/lc37.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Practice driving on Kerbin.</td></tr>
</tbody></table>
<div style="text-align: left;">
The Laythe Rovers are giant 30 ton workhorses. The main function of these 8WD crawlers is to seek out ore to mine and make fuel on Laythe. They each have two large drills, a refinery, and a huge fuel storage tank. They can dock with a parked space plane to refuel it, which is critical for sustaining a link between the Laythe surface and hardware/habitats in orbit.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Landing the rovers is a four-step process. They come packaged in an aero shell with a heat shield, so the initial descent involves just surviving with the heat shield pointed in the right direction. After some time, the drag on the large heat shield flips the package around and the heat shield itself becomes a supersonic air brake, with the aero shell protecting the rover. Once subsonic, the heat shield and fairing are discarded and set of parachutes further slows and rights the rover. Lastly, a set of four rockets slows it to a safe velocity just in time for touchdown.</div>
<div style="text-align: left;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0t3jBiScmsSVbob7NmADLEGCpl9Egipfv42vaA_BzBmPYa1_jeY2PkC5IzsqFn6Qeo9MVLg7oqdjpZvG1g5MnpxVzuvVY2nshXvhp4XVa-0JOAWioNM4Uq30e4-1cBOT6I_L8xobPYO4/s1600/lc38.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="454" data-original-width="1600" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0t3jBiScmsSVbob7NmADLEGCpl9Egipfv42vaA_BzBmPYa1_jeY2PkC5IzsqFn6Qeo9MVLg7oqdjpZvG1g5MnpxVzuvVY2nshXvhp4XVa-0JOAWioNM4Uq30e4-1cBOT6I_L8xobPYO4/s640/lc38.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Step 5 is to quickly deploy the solar panels and drive out of the way of falling fairing debris.</td></tr>
</tbody></table>
<h4 style="text-align: left;">
HAB1</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0LP1y9_1wMiGj9b6ezxSbOhmX4RmCXZcwvJpAx9MvYsUg3IQI1R3hQwxpvkDnc76Mz4z2YlxmQ-y7NBPm57lPrXpmCUJX_psfKuB8Ut_ONDPW8WMIl-z5n6jfjHag4-rKtAgZ8L71Pxw/s1600/lc13.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0LP1y9_1wMiGj9b6ezxSbOhmX4RmCXZcwvJpAx9MvYsUg3IQI1R3hQwxpvkDnc76Mz4z2YlxmQ-y7NBPm57lPrXpmCUJX_psfKuB8Ut_ONDPW8WMIl-z5n6jfjHag4-rKtAgZ8L71Pxw/s640/lc13.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Home, sweet home.</td></tr>
</tbody></table>
<div style="text-align: left;">
The Kerbals can live for extended periods of time in orbit, but having a home base on the Laythe surface will be important for long-term survival. In order to facilitate construction, the surface habitats are themselves rovers with roughly the same chassis as the LR1s. They land the same way and, once on the surface, can drive to each other. This will be important, since the landing target might hundreds of square kilometers.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The habitats are extremely modular. They can be individual homes for a single Kerbal family, including single-passenger mini-rover parked in front. Or, they can be docked together indefinitely to form a larger base, thanks to a central hallway section with docking ports on either end. The slight angle of the hallways allows them to fit inside the aero shell.</div>
<h4 style="text-align: left;">
How much fuel to bring?</h4>
<div style="text-align: left;">
The LR1 and HAB1 landed payloads are both around 28 tons, about half the mass of my <a href="https://scolton.blogspot.com/2014/03/ksp-mission-to-laythe-and-back.html">first Laythe lander</a>. (That lander had to be heavy in order to have enough fuel to get back off of Laythe, a task to be handled by space planes this time around.) In that mission, two identical ships flew independently to Laythe with an average of about 2500m/s of fuel-burning Δv. But, they also made heavy use of <a href="https://en.wikipedia.org/wiki/Aerocapture">aerocapture</a> at both Jool and Laythe. Without that, it would take something more like 4360m/s to get from low Kerbin orbit to low Laythe orbit, according to the amazing <a href="https://forum.kerbalspaceprogram.com/index.php?/topic/87463-13-community-delta-v-map-26-sep-29th/">KSP Subway Map</a>.<br />
<br />
With a known Δv requirement, figuring out how much fuel to bring is simple. With the 800s <a href="https://en.wikipedia.org/wiki/Specific_impulse">specific impulse</a> of the <a href="https://wiki.kerbalspaceprogram.com/wiki/LV-N_%22Nerv%22_Atomic_Rocket_Motor">LV-Ns</a>, the minimum wet to dry mass ratio is:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEho9EDfmsZ_AQz49TaSwuFy4JZqu3CzBnJXhX2bYzbgoGF2xyrPXooCqR0zqaK9-g_cw88TVuY_KFKYo0oHT_8onCR2Hxxya-b5cn17laq_LzjefR7SCWbfHH_cenIgoCPKmxBTtaAxbv0/s1600/lc39.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="171" data-original-width="345" height="99" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEho9EDfmsZ_AQz49TaSwuFy4JZqu3CzBnJXhX2bYzbgoGF2xyrPXooCqR0zqaK9-g_cw88TVuY_KFKYo0oHT_8onCR2Hxxya-b5cn17laq_LzjefR7SCWbfHH_cenIgoCPKmxBTtaAxbv0/s200/lc39.png" width="200" /></a></div>
<div style="text-align: left;">
So, the ships need to carry about 3/4 ton of fuel for every one ton of dry mass. Not too bad, since the same heavy lifter that carries up the packaged landers can carry up an equivalent mass of LV-N engine, fuel tank, and liquid fuel. Thus, each robotic lander requires two separate launches:<br />
<br />
First, a heavy lift booster hauls the lander payload in its aero shell into orbit.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0HF2uYfyZFqsWJ72y3RzPB4hek8Ykmj5evXv4J-b6TLbrKcSPAFqAIW86br6wp7_mBmHGVTBQ8W0XgBy0K3mxXImlCNhQ5nl4Gb5Lyq3dL3UPwWyA9H4_KeM4IUlTDZcwUbIfYCiXRGE/s1600/lc25.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0HF2uYfyZFqsWJ72y3RzPB4hek8Ykmj5evXv4J-b6TLbrKcSPAFqAIW86br6wp7_mBmHGVTBQ8W0XgBy0K3mxXImlCNhQ5nl4Gb5Lyq3dL3UPwWyA9H4_KeM4IUlTDZcwUbIfYCiXRGE/s640/lc25.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">These are the unsung heroes of the mission, relentlessly hauling all the more exciting hardware into orbit.</td></tr>
</tbody></table>
Next, a second booster brings up a propulsion module, with LV-Ns and a lot of liquid fuel.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8BCli-DO2fLVpCyQkzq1HhPYy9lnIOhP9gIdW8Xvs9AYRetJSQeqner4Peru0hbsORPFtPXBnyBkjUbIB2NtRLpGVfm-UWgotmXIszHh9EMxwtxdPAl0WDCYdhwp82QEd6YFvl7VgunU/s1600/lc14.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8BCli-DO2fLVpCyQkzq1HhPYy9lnIOhP9gIdW8Xvs9AYRetJSQeqner4Peru0hbsORPFtPXBnyBkjUbIB2NtRLpGVfm-UWgotmXIszHh9EMxwtxdPAl0WDCYdhwp82QEd6YFvl7VgunU/s640/lc14.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Just remember to check yo' staging...</td></tr>
</tbody></table>
The two meet in orbit and create a transport ship with a wet to dry mass ratio of about 1.775, for a Δv of about 4500m/s.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmET2hwOfsPHswgNSmmru0DsAKFfhZpfk_bfHC28nTUjj41WH0oAOZjZe7QYbsPbRBth5vtD_q-VNvHXy_Eyv310UBGOcb8GdYjZzuFHlFZha3YeYxO17BP-VYVYQYN5V4ZFBMqjEqmG8/s1600/lc40.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="873" data-original-width="1600" height="348" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmET2hwOfsPHswgNSmmru0DsAKFfhZpfk_bfHC28nTUjj41WH0oAOZjZe7QYbsPbRBth5vtD_q-VNvHXy_Eyv310UBGOcb8GdYjZzuFHlFZha3YeYxO17BP-VYVYQYN5V4ZFBMqjEqmG8/s640/lc40.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Rendezvous between an LR1 lander package and its propulsion module.</td></tr>
</tbody></table>
<div style="text-align: left;">
A total Δv of 4500m/s <i>is</i> cutting it a bit close, but they would only need a small amount of aerobraking or <a href="https://wiki.kerbalspaceprogram.com/wiki/Tylo">Tylo</a> gravity assist to gain back a comfortable margin. There's also a good amount of RCS fuel on board that can be dumped (in a prograde or retrograde fashion) near the end of the trip if it's not needed. Additionally, the RS3 ships have a lot of fuel to spare if their propulsion modules can be swapped onto the more thirsty landers nearer to Laythe. The wet to dry mass ratio of the fleet as a whole has a comfortable margin.</div>
<h4>
Launch Window #1</h4>
<div>
The first Jool launch window happens around Day 190 in-game. (The <a href="http://ksp.olex.biz/">simple</a> and <a href="https://alexmoon.github.io/ksp/">more complex</a> online calculators both agree to within a few days). Up to that point, I spent time refining the landers and practicing the landings on Kerbin. But once the designs were locked, the push began to assemble the fleet in orbit. </div>
<div>
<br /></div>
<div>
The practical limit on fleet size is how many ships can be juggled during the actual launch window. In order to boost the wet to dry mass ratio, these ships have the two-engine version of the propulsion module, which gives them a somewhat low thrust to weight ratio. The ~2000m/s ejection burn had to be split into two parts: one into a 10-day elliptical orbit and a second to escape onto the final Jool transfer. Even still, the burns were 10 minutes each, so the ships had to be spaced out so they would reach their final periapsis burn at reasonable intervals.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZqGW1lLqzAHC7w55DDQrmUGN2dbCweLGURdAZT56Tciol3HX4D3UbsUsfX9egEPopqisJtFNhe6GuIiRfeDm0tp4J7MiXaa1RyJzj08P2FSpXVfnYk1XOjp83B3XYnW-tp0GFkAX7PK8/s1600/lc41.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="861" data-original-width="1600" height="344" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZqGW1lLqzAHC7w55DDQrmUGN2dbCweLGURdAZT56Tciol3HX4D3UbsUsfX9egEPopqisJtFNhe6GuIiRfeDm0tp4J7MiXaa1RyJzj08P2FSpXVfnYk1XOjp83B3XYnW-tp0GFkAX7PK8/s640/lc41.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Also, it would be nice if they didn't hit the Mun on their way in.</td></tr>
</tbody></table>
<div>
After the final burn, the transfer takes over two Kerbin years, meaning Kerbin will have been destroyed by the time the first ship even arrives at Jool. The lander designs can never be tweaked, and the crewed fleet will have to set out with no guarantee that there will be a base waiting. No pressure.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij9Btli8kpy50fjrZKZAf_9zGUlLYsn82eKKKuygyc_cuW-zaRzbJpClgeMzrAxLxhpcBtIjHnJtR5O-U-ibWeZGmQGIDhxv_JrLhcUc8z7ywjNpHrPOfPg3JIKxAL9q39WHw991ojUgo/s1600/lc31.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1038" data-original-width="1600" height="415" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij9Btli8kpy50fjrZKZAf_9zGUlLYsn82eKKKuygyc_cuW-zaRzbJpClgeMzrAxLxhpcBtIjHnJtR5O-U-ibWeZGmQGIDhxv_JrLhcUc8z7ywjNpHrPOfPg3JIKxAL9q39WHw991ojUgo/s640/lc31.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Lots of empty space to cross now.</td></tr>
</tbody></table>
<h4>
661 Days Remain</h4>
<div>
With the first 18 ships on their way to Jool and then Laythe to set up base, the priority shifts to getting Kerbals off-planet. This means mass-producing and then filling the immense colony ships, which are the most intricate builds I have attempted in KSP yet.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6TgZmLWSJ_fdeI6it8LgXYXxsNlrLrrOq7WzIzLQVqUFWAdnyPengINsvZ5AFqC1zIbGFDV8_qDdBo7PbosTLBV_jRerOPCHY8Ukg0Z5xmvYMlsSgIFg26mRvrAFo_Iy6NEpzihFnhr4/s1600/lc42.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="715" data-original-width="1600" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6TgZmLWSJ_fdeI6it8LgXYXxsNlrLrrOq7WzIzLQVqUFWAdnyPengINsvZ5AFqC1zIbGFDV8_qDdBo7PbosTLBV_jRerOPCHY8Ukg0Z5xmvYMlsSgIFg26mRvrAFo_Iy6NEpzihFnhr4/s640/lc42.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">More to come...</td></tr>
</tbody></table>
</div>
Shane Coltonhttp://www.blogger.com/profile/10603406287033587039noreply@blogger.com1