1. Unsatisfactory accuracy of recent robotic assisting system ROSA for total knee arthroplasty

Unsatisfactory accuracy of recent robotic assisting system ROSA for total knee arthroplasty

Shin, C., Crovetti, C., Huo, E. et al. (2022)  J EXP ORTOP 9, 82

 

Query Letter to the Editor

Mike B. Anderson MSc1, Louis-Phillipe Amiot MD, MSc, FRCSC2, P Eng., Stéphane Méthot MSc PhD2

1. Zimmer Biomet, Warsaw, IN, USA
2. Zimmer Biomet, Montreal, Quebec, Canada

To the Editor,

We have thoroughly reviewed the recent case series performed by Shin et al.[13] and would like to provide our scientific considerations. We appreciate the authors’ desire to publish on the robotic system and for their work in this arena. The paper attempted to demonstrate the accuracy of the ROSA® Knee System (Zimmer Biomet, Montreal, Quebec, Canada) using 36 inch standing post-operative long-leg, stitched, radiographs (LLRs) compared to their final planned resection angles from the robotic system. We concur with the findings on the accuracy of the system in the coronal plane, where the mechanical axis is simple, well established, and has been supported in several studies [1, 16, 17]. However, a strong measurement bias exists as the validity of the methods used for the sagittal angles is in question. The study compared apples to oranges resulting in unreliable results for both the gamma and delta angles.

The importance of a standardized post-operative radiographic protocol must be stressed as the potential error associated with poor films and the variation associated with different measurement axes, especially in the sagittal plane, exists [5, 9, 10, 18]. This is not the first time that measurement errors in the sagittal plane have been questioned when reporting the accuracy in other total knee arthroplasty (TKA) technologies [5, 7]. At that time, Delport et al.[5] responded with a letter to the editor and encouraged the authors to use “accurate measurement tools” and “compare to the actual plan.” The response was that plain radiographs were standard of care and ethical approval for more advanced imaging was questionable. While this can be respected, experimental studies on accuracy should be held to higher standards than traditional care. While others have recognized the value of LLRs for accuracy assessments in TKA, they provided specific and standardized methods, including the use of a specialized stands to ensure appropriate alignment of the lower extremities or confirmation with advanced imaging[1, 17] in contrast to the study here. There are discrepancies between how the angles in the sagittal plane are calculated within the robotic software and how this study measured them in the post-operative radiographs. This could well explain why the authors see an “unsatisfactory” accuracy in the sagittal plane.

Evaluating the accuracy of a portable navigation device, Shoji et al.[14] noted an outlier rate of 19% for the sagittal femoral component angle and suggested a mismatch in the operative and post-operative axes, and Solayar et al.[16] have recently questioned the validity and reliability of the sagittal femoral angle when measured with post-operative radiographs. In the sagittal plane, the robotic system assesses the femoral mechanical axis as the axis created using the femoral head center and the femoral mechanical axis entry point, based on intra-operative landmarking. The sagittal femoral angle of the femoral distal cut is the angle between the femoral mechanical axis and a line tangential to the femoral distal cut. Shin et al.[13] defined the sagittal femoral angle as “… the point on the head, mid-distal interface of the femoral component to the femoral head tangent.” The mid-distal interface of the femoral component does not necessarily coincide with the femoral mechanical axis. The difference between these two axes (the one computed by the robotic software and the one used in this study) could be exacerbated in a case where a femoral component is malpositioned in the antero-posterior direction. Given the variance in the femoral axes used by the robotic software and the post-operative LLRs in this study, a minimum error of 1-2° is expected [3, 16].

Regarding the mechanical tibial axis in the sagittal plane, the robotic system uses the tibial mechanical axis entry point and the center point of the two malleoli, based on intraoperative landmarking. The tibial slope of the tibial proximal cut is the angle between the tibial mechanical axis and a line tangent to the tibial proximal cut. Shin et al.[13] reported the tibial slope as “…the posterior angle between a line tangent to the tibial component to the dome of the talus“. The exact location of the proximal point at the tibial plateau in the anteroposterior direction used to create the axis of the tibia is not provided and only “Figure 2” is referenced. The paper reports that a mechanical axis was used but based on the figure this does not appear to be the case. Yoo et al.[18] compared five different anatomical axis and noted differences in tibial slope angles of up to 6° based upon the axis used. A rough comparison of “Figure 2” in this paper appears to match the central anatomical axis reported by Yoo et al[18]. As such, a minimal difference of at least 2° between the planned tibial slope and the LLRs would be expected.

In addition to the variation in axes, it is concerning that eight images were considered “faulty,” yet were included with the addition of a third reviewer. Caution should have been used when considering these eight images as inadequate images are prone to errors, and this may have further contributed to differences between the measurements. Several studies have managed such concerns by excluding patients with poor radiographs[9, 16].  Regarding traditional radiographs, Schwarz et al.[12] used a registry database to extract routine LLRs to validate an artificial intelligence algorithm, but they excluded all cases where there was evidence of “incorrect positioning, poor visibility, missing calibration balls or abnormal cropping.” As best practice, the authors should consider excluding the eight faulty cases in their study, or retaking the radiographs, and provided an analysis using the same axes that were used in the robotic planning tools. This method would minimize the measurement bias and provide more sound and reliable scientific evidence.

An overall planning bias is evident. There are several concerning points regarding the understanding of the robotic system and overall methodology. The study noted that no bias was utilized in the selection process; however, the patient attrition in the study is inadequately reported. Patients were excluded for an inability to complete a follow-up visit, but the actual number excluded is unknown. Without transparency on the attrition, it is difficult to consider that a selection bias was not present. Consistent with the Strobe Guidelines[4], the paper should describe the total number of robotic cases performed by the surgeon during the study enrollment period and the final number of cases evaluated, explaining the inclusion and exclusion of cases at each step. This would help ensure that no bias was contrived. It was suggested that the robotic system provides a “best scenario”; however, the system does not provide suggested scenarios but rather demonstrates values according to the surgeon’s decisions. The system does not use the ORTHOsoft Total Knee Navigation System, as suggested. The measurements obtained from the planning software are obtained using software specific to the system. This robotic system was validated using previously validated navigation software[8, 11] which has been found to be more accurate than manual instrumentation when using LLRs[2]. The use of intra-operative alignment measured by the system itself was excluded to avoid “confounding errors”. However, the senior author has previously noted that computer-assisted surgical systems are “the most accurate measurement tool”[6].  Finally, it is uncertain if the power analysis was performed a priori, or even if it is relevant, as the analysis appears to be powered based on range of motion between two separate groups, yet the study reported no such findings and was a comparison of two measurements for the same group.

Despite these concerns and variability associated with the comparison of different axes, both the gamma and delta measurements are within 0.14° of another system’s sagittal measurements[15]. This supports accurate and precise resections were performed. Thus, the use of “Unsatisfactory Accuracy” in the title is a misrepresentation of the data. It is imperative that sound scientific evidence be published, both favorable and unfavorable, for all new technologies in arthroplasty. It is only by these means that we can continue to improve the care provided. We are always open to communications regarding our technologies and the methods used to evaluate and improve them. Given the ramifications of misleading and erroneous research, there should be accountability upon the authors to either correct or redact these findings including amending the title to accurately reflect the results.

Disclosures

All authors are paid employees of Zimmer Biomet. One author (MBA) reports equity interest in OrthoGrid Systems, Inc. and stock options in Zimmer Biomet. One author (LPA) is on the editorial board for the International Journal of Computer-Assisted Surgery and Radiology, and report stock options in Zimmer Biomet.

References

1. Babazadeh S, Dowsey MM, Bingham RJ, Ek ET, Stoney JD, Choong PF (2013) The long leg radiograph is a reliable method of assessing alignment when compared to computer-assisted navigation and computer tomography. Knee 20:242-249
2. Bolognesi M, Hofmann A (2005) Computer navigation versus standard instrumentation for TKA: a single-surgeon experience. Clin Orthop Relat Res 440:162-169
3. Chung BJ, Kang YG, Chang CB, Kim SJ, Kim TK (2009) Differences between sagittal femoral mechanical and distal reference axes should be considered in navigated TKA. Clin Orthop Relat Res 467:2403-2413
4. Cuschieri S (2019) The STROBE guidelines. Saudi J Anaesth 13:S31-S34
5. Delport HP, Vander Sloten J (2015) Evaluation of Patient Specific Instruments. To measure is to know! J Arthroplasty 30:720-721
6. Lionberger DR (2015) Patient specific instrumentation letter to the editor. J Arthroplasty 30:718-719
7. Lionberger DR, Crocker CL, Chen V (2014) Patient specific instrumentation. J Arthroplasty 29:1699-1704
8. Lustig S, Fleury C, Goy D, Neyret P, Donell ST (2011) The accuracy of acquisition of an imageless computer-assisted system and its implication for knee arthroplasty. Knee 18:15-20
9. Maderbacher G, Baier C, Benditz A, Wagner F, Greimel F, Grifka J, et al. (2017) Presence of rotational errors in long leg radiographs after total knee arthroplasty and impact on measured lower limb and component alignment. Int Orthop 41:1553-1560
10. Naendrup JH, Drouven SF, Shaikh HS, Jaecker V, Offerhaus C, Shafizadeh ST, et al. (2020) High variability of tibial slope measurement methods in daily clinical practice: Comparisons between measurements on lateral radiograph, magnetic resonance imaging, and computed tomography. Knee 27:923-929
11. Scholes C, Sahni V, Lustig S, Parker DA, Coolican MR (2014) Patient-specific instrumentation for total knee arthroplasty does not match the pre-operative plan as assessed by intra-operative computer-assisted navigation. Knee Surg Sports Traumatol Arthrosc 22:660-665
12. Schwarz GM, Simon S, Mitterer JA, Frank BJH, Aichmair A, Dominkus M, et al. (2022) Artificial intelligence enables reliable and standardized measurements of implant alignment in long leg radiographs with total knee arthroplasties. Knee Surg Sports Traumatol Arthrosc 30:2538-2547
13. Shin C, Crovetti C, Huo E, Lionberger D (2022) Unsatisfactory accuracy of recent robotic assisting system ROSA for total knee arthroplasty. J Exp Orthop 9:82 https://doi.org/10.1186/s40634-40022-00522-40637
14. Shoji H, Teramoto A, Suzuki T, Okada Y, Watanabe K, Yamashita T (2018) Radiographic assessment and clinical outcomes after total knee arthroplasty using an accelerometer-based portable navigation device. Arthroplast Today 4:319-322
15. Sires JD, Wilson CJ (2021) CT Validation of Intraoperative Implant Position and Knee Alignment as Determined by the MAKO Total Knee Arthroplasty System. J Knee Surg 34:1133-1137
16. Solayar GN, Chinappa J, Harris IA, Chen DB, Macdessi SJ (2017) A Comparison of Plain Radiography with Computer Tomography in Determining Coronal and Sagittal Alignments following Total Knee Arthroplasty. Malays Orthop J 11:45-52
17. Specogna AV, Birmingham TB, DaSilva JJ, Milner JS, Kerr J, Hunt MA, et al. (2004) Reliability of lower limb frontal plane alignment measurements using plain radiographs and digitized images. J Knee Surg 17:203-210
18. Yoo JH, Chang CB, Shin KS, Seong SC, Kim TK (2008) Anatomical references to assess the posterior tibial slope in total knee arthroplasty: a comparison of 5 anatomical axes. J Arthroplasty 23:586-592

 

Response from the authors:

The authors appreciate very much the effort that went into the editorial comments regarding our publication on the ROSA system.

To address this specifically the concerns of the editorial comments, the following delineates the rationale, process and conclusions we followed to respond to these issues we went through including the recalculation of all of our data especially those areas where there was suboptimal performance. The use of specialized stands to ensure the accuracy of sagittal radiographs was not used in our study and we discussed the limitations of our sagittal radiographs in lines 187-195. We suggest that we include additional discussion in regards to the limitations of the sagittal radiographs including what was mentioned in the response to reviewers. That being said, we went beyond the standard of care in repeating these films to perfectly be aligned in the sagittal plane so that exacting calculations could be achieved.

In the second paragraph, an outlier of upwards of 19% for the sagittal femoral component angle is noted, they also suggest malpositioning of the femoral component in the anterior-posterior direction; however this proved to be not true as we have established the accuracy in the coronal plane and would suggest further inaccuracies of the navigational system. Additionally were we to have to translate the femoral component such that it would appear to be malaligned we would have taken that into consideration on our preoperative planning page that we utilized in calculating all of these data points. In other words, if we moved it in any plane other than the expected, it would have appeared in our planning phase. This was the baseline that we utilized to form out our calculation deviation. We utilized the exact same methodology and calculating these angles reported that the computer utilizes to calculate its position of the femoral and tibial component.

Finally, the eight films that were judged to be “suboptimal” were repeated as all patients returned to the office for new radiographs in the sagittal plane. Having completed the calculations on those specific patients we once again reassessed our conclusions. The results are as follows:

• The sagittal femoral angle (gamma) – 60% accurate within 2 degrees, 83% accurate within 3 degrees
  • Tibial slope (delta) – 60% accurate within 2 degrees, 74% accurate within 3 degrees.
  Having performed computer-assisted surgery for total knee replacements for 20 years, I am and will continue to embrace this technology in providing superiority over traditional instrumentation when it comes to accuracy. In the plethora of the literature on
  computer-assisted surgery, one of the most common inaccuracies repeatedly seen is in the sagittal plane. Therefore, it is not unexpected that this might also be the case with robotic surgery. Fortunately it is one of the least important measurements where accuracy is critical in total joint replacement. As such it is entirely possible that the robot
  was perfect in its positioning, however the surgeon (the senior author) was at fault in the stabilization of the block and in the sagittal plane pinning. Therefore, the weak link and
  this is the human interaction which creates its own set of errors remote to that of the servos which control stabilization and movement resistance on the control arm preventing it from maintaining a constant alignment. In performing the surgeries subsequent to this publication I have noticed that it is easy to exceed the resistance of this arm to prevent block skidding thus creating its own set of inaccuracies. This was suggested in our discussion as a potential source of air and again that is in the industry issue that will have to be addressed as human to robot interactions will continue to be a weak link in this technology going forward.
  In conclusion, the intent is not to bash this technology. I embrace it wholeheartedly and look forward to seeing even more improvements as we go forward in refining total knee replacement surgery in the future. As noted in the discussion, there may be room for better instrumentation (such as pin placement and stabilization of the cutting block after robotic positioning) in conjunction with this robotic system that could lessen some of the discrepancies however that is beyond the scope of this paper. Given these updated findings, we do not feel compelled to change our stance on the conclusions given the statistical variants above. Again we would like to thank the scrutiny given by the editorial to make us more cognizant of details which render excellence in achieving accuracy and not only our technology but also are pavers.