ג[i dZddlZddlZddlmZmZmZddl m Z ddl m Z ddl mZddlmZddlmZdd lmZ d#d ed eeefd dfd ZdZde d eeeffdZ d#d ede de fdZ d#d ede de fdZdefdZdeeefdefdZdeeefdefdZdeeeefdefdZdedejdeeeefdefdZ deeefdefdZ!deeeffd Z"d!eeefdefd"Z#dS)$at JSON vs DB Comparison Tab This tab provides a visual comparison interface between JSON files and MongoDB records. Users can review differences, make import decisions, and manually apply imports. Key features: - Load JSON from ODF directory structure - Compare with MongoDB records - Visual diff display with color coding - Manual import decisions - No automatic writes N)DictAnyList) JSONLoader) DiffEngine)ImportPreviewEngine) UpdateEngine) DataService)ObjectId data_serviceconfigreturnc|dd}tjdtjd|stjddSt ||d}t |d}t}ttjdt|||tjdtj d r2tj d rt|||||tj d r&tjdt||dSdS) z Render the JSON vs DB comparison tab. Args: mongo_service: MongoService instance data_service: DataService instance config: Application configuration dictionary json_sources base_pathu🔄 JSON vs MongoDB Comparisona Compare JSON files with MongoDB records and manually import selected data. **Workflow:** 1. Select collection and JSON source 2. Load and compare data 3. Review differences 4. Make import decisions 5. Apply selected imports u7❌ Not connected to MongoDB. Please check the sidebar.N collections---comparison_collectioncomparison_sourcecomparison_results)sttitlemarkdown is_connectederrorrrr_init_comparison_state_render_selection_panel session_stateget_render_load_panel_render_results_panel) mongo_servicer r settings_servicejson_base_path json_loader diff_enginepreview_engines ./var/www/html/IGF-ODF-V3/ui/tabs/json_vs_db.pyrender_json_vs_db_tabr)sN+K8NH .///K       % % ' ' JKKK^VM-BCCKVM233K(**NKM;???K 344 9I9M9MNa9b9b           011= Enm<<<<<==c~dtjvrdtj_dtjvrdtj_dtjvrdtj_dtjvrdtj_dtjvrdtj_dtjvrd tj_dSdS) z,Initialize session state for comparison tab.rNrrcomparison_loadedFlast_comparison_keydata_editor_keyr)rrrrrr,r-r.r*r(rrZsb&66615.""222-1*2#333.2+""223-2*B$444/3, 00-+,(((--r*r%cjtjdtjd\}}|5|}|r*tjd|dd}|tj_n/tjd|jdtj_d}dddn #1swxYwY|5|d }d | D}|r9g} |D]3} | | |} | d kr| | 4ng} | rdtjd | d d} | tj_ | | } | drtjd| dn-|rtjd|ddtj_ d} dddn #1swxYwY| dtjj}tjj|kr3dtj_dtj_|tj_| r tjjr|| tjj}|drtjd\}}}|5tjd|ddddn #1swxYwY|5tjd|ddddn #1swxYwY|5|| }tjd|dddddS#1swxYwYdSdSdSdS)z Render collection and source selection panel. Args: mongo_service: MongoService instance json_loader: JSONLoader instance config: Application configuration dictionary u$1️⃣ Select Source and Collectionz JSON Sourcecomparison_source_selectorz;Select JSON source directory (e.g., tokyo_2020, paris_2024))optionskeyhelpu ⚠️ No JSON sources found in Nrcpg|]3\}}|dd|dd1|4S)enabledTallow_manual_importFr).0name coll_configs r( z+_render_selection_panel..sX   &T;y$// 4?OODY[`4a4a    r*r Collectioncomparison_collection_selectorzGSelect a collection (only showing collections with data in this source)validation_enabled📋 descriptionu$⚠️ No collections with data in ``_F supportedz ODF Folderfolderz JSON Files file_countz MongoDB Docs,)r subheadercolumnsget_available_sources selectboxrrwarningritemscount_files_in_sourceappendrget_validation_infoinfor-rr,get_collection_infometriccount_documents)r"r%r col1col2available_sourcesselected_sourcecollections_configall_collectionsavailable_collections coll_namerHselected_collectionvalidation_info current_keyrScol3db_counts r(rrosL7888AJD$ ##'==??  # l)0R O 2AB  . . JQ+:OQQ R R R15B  ."O###############" %'%'#M2  *<*B*B*D*D     '$& !, < < (>>y/ZZ ><)00;;; < %' !  '"$,-4^ ###  6IB  2*==>QRRO34 B@ >@@AAA V T/TTTUUU59B  2"& K%'%'%'%'%'%'%'%'%'%'%'%'%'%'%'P)OO2+;+MOOK +{:;.2+-2*/:,;r/A;..    .     ;!z!}} D$ 8 8 ,X777 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 < < ,\(:;;; < < < < < < < < < < < < < < < ; ;(889LMM .X//::: ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;;;;; ; ;s[A0B**B.1B.7C7F::F>F>>J&&J*-J*3KK"K(-L""L&)L&r&cFtjdtjddg\}}|5tjddddn #1swxYwY|5tjddd rt |||||ddddS#1swxYwYdS) a Render load and compare panel. Args: mongo_service: MongoService instance data_service: DataService instance json_loader: JSONLoader instance diff_engine: DiffEngine instance settings_service: SettingsService instance (optional) u2️⃣ Load and ComparerFzFClick 'Load & Compare' to analyze differences between JSON and MongoDBNu🔄 Load & Compareload_compare_btnprimaryr4type)rrJrKrSbutton_perform_comparison)r"r r%r&r#rWrXs r(r r sZ"L+,,,QF##JD$ ZZ XYYYZZZZZZZZZZZZZZZ    9 !"                          s#AAA+BBBc tjj}tjj}tjd5|||}|rt d|D}t|dkr9tjd|tjd ddddS||vr-tj d| d|d d } |r| } | || } dddn #1swxYwYtjd 5| ||| } dddn #1swxYwYd} |r||} | rktjd 5ddlm}ddlm}|tjj}|}|||| } dddn #1swxYwY| tj_| tj_dtj_tjxjdz c_dtjvr tj`dtjvr tj`t }|D]3}|d|D4|r|dd|n|}t=d|D}||}| dd}| ddd}| ddd}| dg}tj d!gd|d|d |rd!!|nd"d#|d$| d%d&|d'd!!| d(d)|rd!!|nd*d+| d%d,d-| d.d,d/| dd0d,d1| dd2d,d3|d4|d5| dd6d,d7| dd8d,d9t|d,d:| rf| d}|d;dkr(tjd<|d=d,d>|d?d,d@|d;d,dA|dBdC tj"dDdE5tG| dFdD]\}}|dG}|dH}|dI|dJdK|}tj$dL|dM|dN|D]1} tjdO| dPdQ| dRdS| dT2tj$dU dddn #1swxYwYnTtj dV|d=d,dA|dBdWn+|r)||rtj%dXtj"dYdZE5tj$d[tj&d\t|tj&d]t| tj&d^| r1tOd_| dDnd`|rgtj$dadb|dDddc}!tj(d!!|!| rgtj$ddde| dDddc}"tj(d!!|"tj$dftj&dg| d(tj&dh|tj&di||rs| d(rktj$dj|d}#tj&dk| d(D]2}$|#|$dl}%tj&dm|$dn|%3| r_| d(rWtj&do| d}&| d(D]2}$|&|$dl}%tj&dm|$dn|%3dddn #1swxYwY|dpkrI|rGtj%dqd!!| d(drd!!|ds|| d%krtj dttj)dS)uz Perform the comparison between JSON and MongoDB. Args: mongo_service: MongoService instance data_service: DataService instance json_loader: JSONLoader instance diff_engine: DiffEngine instance zLoading data...c3@K|]}|dVdS)_target_collectionNr9r:rs r( z&_perform_comparison..s/$W$WQQUU+?%@%@$W$W$W$W$W$Wr*reu7⚠️ ERROR: Mixed collections detected in JSON data: zBThis should not happen. Each collection must be loaded separately.Nu8⚠️ WARNING: JSON records are marked for collection 'z' but you selected ''i)limitzComparing records...z'Validating foreign key relationships...r)ForeignKeyValidator) MongoServiceTnew_records_preview_dfupdate_preview_dfc3DK|]}|d|VdS)rDN startswithr:ks r(rqz&_perform_comparison..Ts3QQQq||C?P?PQqQQQQQQr*rnc3FK|]}|dddVdS_validation_passedFreNr9ros r(rqz&_perform_comparison..Zs7!`!`QUUCWY^=_=_!`!!`!`!`!`!`!`r*matching_strategy primary_onlysummaryprimary_key_matchesalternative_key_matchesalternative_keysuD ✅ **Comparison Complete** - **Target Collection:** `z` - **JSON Source:** `z%` - **Expected Document Types:** , rz - **Validation:** / json_countzG records passed document type validation - **Matching Strategy:** `z` - **Primary Keys:** matching_keysz - **Alternative Keys:** Nonez - JSON Records: rIz - MongoDB Records: rcz - Exact Matches: exact_match_countz - Duplicates: duplicate_countz (Primary: z, Alternative: z) - New Records: new_record_countz - New Fields: new_field_countz - Total JSON Fields: z# (all nested fields flattened) invalid_countui ❌ **Foreign Key Validation Failed** - **Total Records Checked:** total_recordsz" - **Valid Records:** valid_countz$ - **Invalid Records:** z) - **Foreign Keys Checked:** foreign_keys_checkedur ⚠️ Some records have invalid foreign key references. See details below. u"❌ Invalid Foreign Key References)expandedinvalidrecorderrorscodeidRecord #z**z . Record: `z`**u ❌ Field `fieldz` = `valuez`: rruc ✅ **Foreign Key Validation Passed** - **Records Checked:** zL All foreign key references are valid! uBℹ️ Foreign key validation is configured but was not performed.u🔍 Debug InformationFz### Data Loading DetailszJSON records loaded: zMongoDB records loaded: zMongoDB records are flattened: c3K|]}d|vV dS).Nr/r{s r(rqz&_perform_comparison..s&5]5]1cQh5]5]5]5]5]5]r*N/Az+### Sample JSON Record Keys (first record):c<g|]}|d|SrDryr{s r(r=z'_perform_comparison..s*[[[aVYIZIZ[[[[r*z.### Sample MongoDB Record Keys (first record):c<g|]}|d|Srryr{s r(r=z'_perform_comparison..s*WWWAQ\\RUEVEVWaWWWr*z### Business Key MatchingzPrimary keys configured: zAlternative keys configured: zMatching strategy: z### Sample Business Key ValueszFrom first JSON record:z NOT FOUNDz : zFrom first MongoDB record:primary_or_alternativeu\💡 **Multi-Key Matching Enabled:** - System first tries to match by primary keys: z0 - If no match, tries alternative keys: zj - This allows matching athletes across different competitions even if their codes differ uc⚠️ Note: Some files were skipped due to document type mismatch. Check console logs for details.)*rrrrspinnerload_collection_from_sourcesetlenrrNpopget_record_limitload_flattened_recordscompare_collectionsget_foreign_keysservices.foreign_key_validatorrtservices.mongorur"validate_foreign_keysrfk_validation_resultsr,r.rvrwupdatekeysrsumget_expected_document_typessuccessjoinexpander enumeraterrStextanyrrerun)'r"r r%r&r#collection_name source_name json_recordstarget_collections record_limit db_recordsresultsr foreign_keysrtru fk_validatorall_records_to_validateall_json_fieldsrtarget_collectionvalidation_passed_countexpected_typesrprimary_matches alt_matchesr fk_summaryidxitemr record_idrsample_json_keyssample_db_keys sample_jsonr4r sample_dbs' r(rkrks &>     X!$$W$W,$W$W$W!W!W %&&* gSegghhh]^^^        &88 X WVhVlVlVnVnWWETWWWXXX  ?+<<>>L!88 9  3               < * + +  11                    !'88II  EFF  NNNNNN7777772223C3QRR +7'(4(J(J#+ ))%               +2B'-BB*)-B&$$)$$ 2#334   3b../   .eeORRQQ&++--QQQQQQQWcw Q++,@/RRRhw"!`!`\!`!`!```!<<=NOON $7HHi(,,-BAFFO)$(()BAFFK{{#5r::J 1%  BP$ZDIIn$=$=$=UZ  /  291F  17?#;<<=MXTYY'7888RX\* +i()<=9%&78HWhsI&'9:9%&78 o..!   (%V*95 o & *!  H +5_*E   #-]";    %/$?    *44J)K      ADQQQ ' '!*+@+KQ!O!O ' 'IC!(^F!(^F & 66::dDTsDTDT3U3U V VIK CS C CY C C CDDD!'ii!guW~!g!gE'N!g!gW\]dWe!g!ghhhhK&&&& ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' J%/$?*44J)K      V.??PPV TUUU - > > >"-"- ./// ;L(9(9;;<<< <3z??<<=== xak2v#5]5] ST HZHZH\H\5]5]5]2]2]2]qvxxyyy  1 KE F F F[[<?+?+?+A+A[[[\_]_\_`  GDII.// 0 0 0  / KH I I IWWA););)=)=WWWX[Y[X[\N GDIIn-- . . . /000 FGO,DFFGGG B0@BBCCC 9&799:::  -GO4 - K8 9 9 9&q/K G- . . ./ - -#[99+S++E++,,,,  -'/2 - G0 1 1 1"1 I/ - -! c;77+S++E++,,,,E"-"-"-"-"-"-"-"-"-"-"-"-"-"-"-J44 9I   8< '/BZ8[8[  04 :J0K0K      !66{ yzzzHJJJJJsdA1D!5A D!!D%(D%E$$E(+E(AG**G.1G.1B=V;;V?V?1J.c++c/2c/r'ctjj}tjdtjd\}}}}|5tjd|ddddddn #1swxYwY|5tjd |dd |dd d d dddn #1swxYwY|5tjd|dd|ddd d dddn #1swxYwY|5tjd|ddddddn #1swxYwYtjdtjd|dddd|dd dd|dddd|dddg\}}} } |5t||dddn #1swxYwY|5t|||dddn #1swxYwY| 5t|||dddn #1swxYwY| 5t|ddddS#1swxYwYdS)z Render comparison results with tabs. Args: preview_engine: ImportPreviewEngine instance mongo_service: MongoService instance u3️⃣ Review Differencesz Exact MatchesrrzRecords that are identicalr5N Duplicatesrnormalz8Records with matching business keys but different values)delta delta_colorr5z New RecordsrzRecords not found in MongoDB New FieldsrzFields in JSON not in MongoDBru✅ Exact Matches ()u⚠️ Duplicates (u🆕 New Records (u📋 New Fields () rrrrJrKrUrtabs_render_exact_matches_tab_render_duplicates_tab_render_new_records_tab_render_new_fields_tab) r'r"rrWrXrbcol4tab1tab2tab3tab4s r(r!r!s1GL-... Z]]D$d     I 2 3-                        I 0 1)$%67 K                         I 1 2)$%78 /                         I 0 10                   K WHgi01DEHHHFgi01BCFFFFWY/0BCFFFDGI./@ADDD &D$d ;;!'>:::;;;;;;;;;;;;;;; GGw FFFGGGGGGGGGGGGGGG HHGGGHHHHHHHHHHHHHHH ((w'''((((((((((((((((((s$A11A58A5>2B<<CC 2DD D $EE EGG#&G#,H  HHH55H9<H9II#&I#rc|d}|stjddStd|D}tjdt |d|t |krtjdtjd||d }|js|d |j Dd }tjd t |dt |j dtjdtj |ddddSdS)z%Render exact matches tab (read-only). exact_matchesuℹ️ No exact matches foundNc3FK|]}|dddVdSr~r9ros r(rqz,_render_exact_matches_tab..#s7YYaee.6s)%Q%Q%QAq||C?P?P%Qa%Q%Q%Qr*ignore)rKr📊 Displaying  records with z% fields (all nested fields flattened)u<**🟠 These values are identical in both JSON and MongoDB**stretchTwidth hide_indexheight) rrSrrrrbuild_preview_dataframeemptydroprK dataframe)rr'rvalidated_countdf display_dfs r(rrsO,M  /000YY]YYYYYOJTc-((TTTUUU#m,,,D BCCCK     / / w G GB 8  WW%Q%Q%Q%Q%QZbWcc  A3z??AA#jFXBYBYAAA B B B RSSS           r*c|d}|stjddStd|D}td|D}td|D}tjdt |d|t |krtjd n(tjd |d t |d |d krbtjd|dd|dd|dd|dgd n1tjdd|ddtjdtjdt|D]\}}|dd} |dd} | dkrdnd } | dkrd!nd"} tj | d#|d$zd%| d&| 5tj d'| tj d(| dddn #1swxYwYtjd)| |j sfd*} tjd+j | d$,}tjd-t d.tj|d/d0d12tjd3tjd)t!||d4|dSdS)5z+Render duplicates tab with comparison view. duplicatesuℹ️ No duplicates foundNc3RK|]"}|ddddV#dS json_recordrFreNr9r:ds r(rqz)_render_duplicates_tab..Qs>ee1]3C3G3GH\^c3d3de!eeeeeer*c3LK|]}|ddkdV dS) match_method primary_keyreNr9rs r(rqz)_render_duplicates_tab..Ts8ZZ1553H3HM3YZ!ZZZZZZr*c3LK|]}|ddkdV dS)ralternative_keyreNr9rs r(rqz)_render_duplicates_tab..Us9ZZAquu^/D/DHY/YZaZZZZZZr*⚠️ z potential duplicate(s) foundu9✅ All duplicate records passed document type validation ⚠️ Only rz duplicates passed validationru"🔍 **Match Methods:** - z matched by primary keys: rrz - z matched by alternative keys: ru 💡 Alternative key matches allow finding the same athlete across different competitions even when their athlete code differs. z1 These records match by business keys: **z ** u9 **Update Workflow:** 1. Review field-by-field comparison below 2. Select which fields to update (✅ Update or ❌ Skip) 3. Apply updates to MongoDB 💡 **Use Case:** When athlete info changes (name, birthdate, nationality), you can selectively update only the changed fields. z### Duplicate Recordsrunknown matching_keyrr u🔑u🔍z Primary KeyzAlternative Keyz Duplicate #rez - Matched by rzMatch Method: zMatching Values: rcg}tjD]\}}|dkr|d!|dkr||ddkr|dI|ddkr|dk|ddkr|d |d|d kr|d |d kr|d |d|S)Nrrstatusnewz,background-color: #90EE90; font-weight: bold differentz,background-color: #FFD700; font-weight: boldmissing_in_jsonz,background-color: #FFB6C1; font-weight: bold json_valuez9background-color: #E3F2FD; border-left: 3px solid #2196F3db_valuez9background-color: #FFF3E0; border-left: 3px solid #FF9800)rrKrQ)rowstylesrcolcomp_dfs r(highlight_comparisonz4_render_duplicates_tab..highlight_comparisons=F%go66 & &S'>&MM"%%%%H_&8}-* &TUUUUX+5* &TUUUUX*;;* &TUUUU b))))L(&MM"]^^^^J&&MM"]^^^^MM"%%%%Mr*z### Field-by-Field Comparison)axisrz- field comparisons with full nested structurerTrru **Color Legend:** - 🟢 **Green** = New field in JSON (not in MongoDB) - 🟡 **Yellow** = Different values between JSON and MongoDB - 🔴 **Pink** = Field exists in MongoDB but missing in JSON - 🔵 **Blue background** = Value from JSON file - 🟠 **Orange background** = Value from MongoDB - ⚪ **White** = Matching values r)rrSrrNrrrrrrr$build_duplicate_comparison_dataframerstyleapplyr_render_update_controls)rr'r"rrrrrduprr match_icon match_labelr styled_dfrs @r(rrDs@ &J  ,---eeZeeeeeOZZZZZZZZOZZZZZZZKJGZGGGHHH#j//)d LMMMM b/bbC OObbbcccQ      6:ii@X6Y6Y    6:ii L^`b@c@c6d6d         157?;S1T1T      K   K'(((j)) 8 8Sww~y99 ww~u55 +}<HVV& '3}'D[mmJ[ [JhhC!Ghh;hhZfhh i i 8 8 G2[22 3 3 3 G6 66 7 7 7 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8KAA*MMG =:      0 3444M''(<1'EE  ^3w<<^^^___            E  % &      m: : s/II I rrc tjdtjdt}dtjvr$||tj_tjj}|jrtjddStjgd\}}}}} |5tj ddd r:| |d g tj_tj dddn #1swxYwY|5tj d dd r:| |dg tj_tj dddn #1swxYwY|5tj ddd r7| |tj_tj dddn #1swxYwY| 5tj ddd r7||tj_tj dddn #1swxYwYtj d||} tjd\}}}}|5tjd| ddddn #1swxYwY|5tjd| ddddn #1swxYwY|5tjd| ddddn #1swxYwY|5tjd| d dddn #1swxYwYtj dtj d!tj d"tj|d#d$d%d&tjd'd$d()tjd*d$d()tjd+d$d,)tjd-d$d()tjd.d$d,)tjd/d$d,)tjd0d1d2gd$d3d(4d5d6tjj7} | tj_| dd8krtj dtjd9| dd:| dd;tjdd?d@Art/|| |||ddddS#1swxYwYdSdS)Bz Render update controls for selective field updates. Args: duplicates: List of duplicate records collection_name: Name of the collection mongo_service: MongoService instance u🔄 Selective Field Updatesu 💡 **How it works:** - Select which fields you want to update in MongoDB - New fields are selected by default - Different fields require your decision - Matching fields are skipped by default rwuℹ️ No fields to updateN)r1rererereu✅ Update New Fieldsupdate_new_fieldszMark all new fields for update)r4r5r) status_filteru✅ Update Differentupdate_different_fieldsz$Mark all different fields for updateru ❌ Skip Allskip_all_updateszSkip all updates 🔄 Resetreset_update_decisionszReset to defaultsrrzRecords to Updaterecords_to_updatezFields to Updatefields_to_updater new_fieldszDifferent Fieldsdifferent_fieldsz### Select Fields to UpdateuR **Color Legend:** - 🟢 **Green** (new) - Field exists in JSON but not in MongoDB - 🟡 **Gold** (different) - Field value differs between JSON and MongoDB - 🟣 **Pink** (missing_in_json) - Field exists in MongoDB but not in JSON - 🔵 **Light Blue** - JSON value - 🟠 **Light Orange** - MongoDB value rTfixedrrsmall)disabledrzDB IDz Field NamemediumStatusz JSON Valuez MongoDB ValueActionskiprz#Choose whether to update this fieldr3requiredr5r) _record_index_db_idrrrr_update_decisionupdate_preview_editor_rrnum_rowsr column_configr4ru⚠️ Ready to update z record(s) with z field(s)rFreu🚀 Apply Updatesapply_updates_btnrgrh)rrJrSr rbuild_update_preview_dataframerwrrKrjmark_all_for_updatermark_all_for_skiprget_update_summaryrU data_editorr? NumberColumn TextColumnSelectboxColumnr.rN_apply_field_updates) rrr" update_enginerrWrXrbrcol5r edited_dfs r(r r seL/000G    !NNM""22f-:-Y-YZd-e-e*  +B x ,---$&:ooo#>#> D$dD  9,2ELl m m m 1>1R1RSUfkel1R1m1mB  . HJJJ  9+1JQw x x x 1>1R1RSUfqer1R1s1sB  . HJJJ  9^);BT U U U 1>1P1PQS1T1TB  . HJJJ  9\'?FY Z Z Z 1>1]1]^h1i1iB  . HJJJ K..r22GZ]]D$d EE %w/B'CDDDEEEEEEEEEEEEEEE CC $g.@&ABBBCCCCCCCCCCCCCCC 77 , 5666777777777777777 CC $g.@&ABBBCCCCCCCCCCCCCCCKK-...K    -::; &112 %001 &112 *556 (334 !# 0 @ @*: !A!!?& & N HR%5%E G G[...Ib*3B&!"Q& E BW5H-IBB[bcu[vBBB C C CZA'' d   y$'  %#!!                   s5ADDD AE>>FF AG&&G*-G*3AIIIKK  K K::K>K>L//L36L3<M$$M(+M(7+U//U36U3 preview_dfrJc tjd||}tjd|dd|d|dd|dd |d d tjd \}}|5tjd dddr\|||}|s"tjd ddddS||} | r tj dt|d5d} g} |D]} dt| ddi} | | | d}|j dkr| dz } M#t$r8}| d| ddt!|Yd}~d}~wwxYw dddn #1swxYwY| dkrtjd| d|d | rItjd!t| d"| dd#D]}tjd$|dtj_dtj_d%tj_tjd&n/tjd'| r| D]}tj|nO#t$r.}tjd(t!|Yd}~nd}~wwxYwtjd)dddn #1swxYwY|5tjd*d+d,r'tjd-tjddddS#1swxYwYdS).a2 Apply selected field updates to MongoDB. Args: collection_name: Name of collection preview_df: Preview DataFrame with update decisions duplicates: Original duplicate records update_engine: UpdateEngine instance mongo_service: MongoService instance z### Confirm Updatesu& ⚠️ **You are about to update r,z record(s) in z+** This action will: - Update r-z field(s) in MongoDB - Preserve all other fields unchanged - Cannot be undone automatically **Fields to update:** - New fields: r.z - Different fields: r/z r1u✅ Confirm Updatesconfirm_updates_btnrgrr4riru#❌ No update operations to performNz Updating z record(s)...r_idfilterrrezRecord record_indexru✅ Successfully updated z record(s) in collection: `rCr z update(s) failed:z - F1💡 Reload the comparison to see updated resultsu⚠️ No records were updatedu❌ Failed to update: u❌ Collection not found ❌ Cancelcancel_updates_btnr4rzUpdates cancelled)rrrDrNrKrjprepare_update_operationsrget_collectionrrr update_onemodified_count ExceptionrQstrrrrrwrr,rSr)rrMrrJr"rrWrX operations collection updated_countrop filter_dictresulters r(rIrIms&"K%&&&..z::GJ &-.A&B  Ra  ()  <(  !!34       AJD$ 9595 9 !%    8 5'@@ZXXJ >???9595959595959595'55oFFJ( 5%@$NJ$N$N$NOO X X() !#",XXBX/4hr(|E?R6S6S.T )3)>)>{BxL)Y)Y#)#81#<!7$1Q$6M#,XXX & .V>8J.V.VcRSff.V.V W W W W W W W WXX X X X X X X X X X X X X X X X%q(0 #{}#{#{ix#{#{#{|||!8J'PV'P'P'PQQQ)/88 "u 7 7 7 7>B(:>B(;=B(: STTTT #CDDD!0)/00 " @@@H>c!ff>>????????@3444s959595959595959595959595959595v  9 $     G' ( ( ( HJJJs?ALL,%J& F7A E%$F7% F' /.F" F7"F' 'F7+ J&7F; ;J&>F; ?C&J&%L& K0$KLKLLL?MM!Mc  |d}|stjddS|r"|ddddnd}td|D}tjd t |d |d |t |krtjd |d n(tjd|dt |dtjddtjvr%| |dtj_ tjj }|j rdStj gd\}}} } |5tj dddr7||tj_ tjdddn #1swxYwY| 5tj ddr7||tj_ tjdddn #1swxYwY| 5tj ddr8| |dtj_ tjdddn #1swxYwYtjdd|j D} tjd t |d!t | d"tjd#gd$d%d&d'(tjd)d%d*d'+d,} |j D]} | d-s|| j}|d.vr%tj| d/0| | <M|d1kr%tj| d20| | <xtj| d30| | <tj|dd%d4d5| d6tjj7}|tj_ ||}tjd8tj d9\}}} |5tjd:t |d;dddn #1swxYwY|5tjdt |d?dddn #1swxYwYt |d;dkrtjdtjd@t |d;dAdBtjvrdtj_tj d9dCg\}}|5tj dDdEdFr5|dG|d;||dHtj_tjdddn #1swxYwYtjj$t;tjj||dSdSdS)Iz,Render new records tab with import controls. new_recordsuℹ️ No new records foundNrrrnrc3RK|]"}|ddddV#dSrr9ros r(rqz*_render_new_records_tab..s>hh!a 6F6J6JK_af6g6ghAhhhhhhr*u🆕 z0 new record(s) ready for import to collection: `rCu✅ All z( records passed document type validationr rz records passed validationu These records are not in MongoDB. You can: - Review and edit the data - Select records to import - Apply imports manually 🔵 **Blue background** = Values from JSON files (not yet in MongoDB) ✅ **Validated** = Document type matches target collection rvr)r1rerereu✅ Import Selected Recordsrgr)riru❌ Mark All Skip mark_all_skip)r4r*reset_decisionsrc<g|]}|d|Srryrs r(r=z+_render_new_records_tab..s)CCC!c1B1BCACCCr*rrzS data fields (all nested fields flattened). Scroll horizontally to see all columns.zImport?)pendingimportr6Tz$Select whether to import this recordr1r7Rowz Row index)r2r5r)_import_decision _row_indexrD)int64int32float64float32zNumeric value from JSON filerboolzBoolean value from JSON filezValue from JSON filer0inew_records_editor_r=z### Import SummaryrFz To ImportrmzTo Skipr6Pendingrlu⚠️ Ready to import z record(s)pending_importreu🚀 Apply Importsapply_imports_btnrhr)rimport_indicesrgrM)rrSrrrrrNrrrrvrrKrjmark_all_for_importrrCr?rHrFrzdtypeCheckboxColumnrGrEr.extract_import_decisionsrUrx_show_import_confirmation)rr'r"rgrvalidated_recordsrrWrXrbr data_columnsr?rr|rL decisionss r(rrs -(K  -..._jx A}599:NPYZZZoxhh{hhhhhJms;''mmYjmmmnnnC ,,,d V,VVVWWWW b"3bbc+6F6FbbbcccK    r'77 2@2X2X  3 3 /  0B x Z 55D$d  92) T T T 6D6X6XY[6\6\B  3 HJJJ  9(o > > > 6D6V6VWY6Z6ZB  3 HJJJ  9\'8 9 9 9 6D6\6\77B  3 HJJJ KDCrzCCCLG ]s2ww ] ]c,6G6G ] ] ]^^^,<< 1117 =  &33  4  M"z~~c"" sGME@@ %'%5%B%B8&C&& c""& %'%5%D%D8&E&& c"" &(%5%@%@0&A&& c"  # D""2"B D DI/8B+77 BBIK$%%%z!}}D$ 99 +s9X#677888999999999999999 55 )S6!233444555555555555555 88 )S9!566777888888888888888 9X !#  E QS81D-E-EQQQRRR 2#3 3 3.2B  +ZA'' d   y$' (//@'A&/&9#."+ 33 /                    *  % /     9  6  sAF55F9<F9AHH #H )AJJ J)R  RR)SSS)TTT/A XX X c|d}|stjddStjdt|dtjd|D]}tjd|tjddS) zRender new fields tab.r.uℹ️ No new fields detectedNrAz new field(s) found in JSONz These fields exist in JSON but not in MongoDB. **Recommendation:** Use the Schema Manager to add these fields before importing. u• uH💡 Go to the collection tab and use Schema Manager to add these fields)rrSrNrrr)rr.rs r(rrs&J  /000JCs:CCCDDDK      uG VWWWWWr* pending_datacr|d}|d}|d}|d}tjdtjdt|d|dtjd \}}|5tjd d d d r&g} |D]L} t | tr d| vr| | d7| | M| r| d d|n|} | || |d} t| dkr"tjd ddddS| | } | &tjd| d ddddS tj dt| d| d5| | }dddn #1swxYwYtjdt|jd| ddtj_dtj_dtj_dtj_tjdtjn;#t.$r.}tjd t1|Yd}~nd}~wwxYwdddn #1swxYwY|5tjd!d"d #r8dtj_tjd$tjddddS#1swxYwYdS)%a  Show import confirmation dialog and execute import. Args: pending_data: Dictionary with collection_name, import_indices, new_records, preview_df preview_engine: ImportPreviewEngine instance mongo_service: MongoService instance rrzrgrMu### ⚠️ Confirm Importz **You are about to import z record(s) to `zp`** This action will: - Insert new documents into MongoDB - Cannot be undone automatically r1u✅ Confirm Importconfirm_import_btnrgrrPrrrnT) unflattenu#❌ No records prepared for import!Nu❌ Collection `z%` not found or MongoDB not connected!z Importing z records to z...u✅ Successfully imported z records to collection: `rCFrUu❌ Failed to import: rVcancel_import_btnrXzImport cancelled)rrrrrKrj isinstancedictrQrprepare_records_for_importrZr insert_manyr inserted_idsrrvrr,rxrSballoonsr]r^r)rr'r"rrzoriginal_recordsrMrWrXrrecrrecords_to_importr`rdres r(rrs##45O!"23N#M2l+JK+,,,H">22CR   AJD$ 6<6< 9 $    5 <L' - -c4((-]c-A- ''M(:;;;; '',,,,_k  Q 3 34H/ Z Z Zp !/ I I !J!! $%%* >???;6<6<6<6<6<6<6<6<@'556GHHJ d,=dddeeeI6<6<6<6<6<6<6<6rsY  """"""""""++++++++++++>>>>>>//////------ ====== cN==  ========@---*^; ^;DQTVYQYN^;^;^;^;L $$$$ $$$$X ZZZZ ZZZZzC(*=C(C(C(C(L' tCH~' GZ' ' ' ' T #s(^ '    DdT#s(^$dddddNgg gT#s(^$g ggggTp #s(^p'ppppfXDcNXXXX.^sCx.^'^^^^^^r*