x-algorithm changelog (unofficial)

Unofficial, community-maintained project. Not affiliated with, endorsed by, or operated by X Corp. or xAI. Notes are generated from the public git history of xai-org/x-algorithm and may contain mistakes; the upstream diff is the source of truth.

· aad7179 · 79 files, +3,303/−651

New filter following_viewer_muted_keyword_filter and other changes.

candidate hydratorsconstantsfiltersnumeric tweaksschema
  • New candidate hydrator: quoted_post_text_hydrator.
  • New filter: following_viewer_muted_keyword_filter.
  • Renamed: muted_keyword_filter → viewer_muted_keyword_filter.
  • Constants changed: MULTIPART_PART_SIZE 16 * 1024 * 1024 → 32 * 1024 * 1024 (checkpoint_store.rs).
  • Other literal values changed in code: mode 4.5→4.6 (classifier.py), learning_rate 1e-3→7.1e-4 (xrecsys.py), static_assert(sizeof(UpdateScalars) == 4 * sizeof( 3→4 (async_emb_kernel.hpp), input_output_aliases 12→14, 13→15, 14→16 (ranker_attention_varlen_fa4.py), input_output_aliases 12→14 (ranker_attention_varlen_fa4.py) and 2 more.
  • Proto/Thrift schema: 8 fields added (e.g. FetchAggregatedUserActionRequests.feature_switch_overrides, NextActionDistribution.rewardOutputs, RewardOutputs.policyLogits, RewardOutputs.values and 4 more).
  • Most activity in phoenix/ (+2,811/−536), home-mixer/ (+222/−25), grox/ (+146/−88) and visibility-filtering/ (+124/−2).

Full analysis → Summary: automatic analysis

Config parameters (param! defaults)

None.

Pipeline components

KindChangeFile
Candidate hydratoraddedhome-mixer/candidate_hydrators/quoted_post_text_hydrator.rs
Filteraddedhome-mixer/filters/following_viewer_muted_keyword_filter.rs
Filterrenamed from muted_keyword_filterhome-mixer/filters/viewer_muted_keyword_filter.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/conversation_gap_ancestor_hydrator.rs
Candidate hydratormodifiedvisibility-filtering/hydration/tes_hydrator.rs

Changed constants & literal values

NameOldNewFile
MULTIPART_PART_SIZE16 * 1024 * 102432 * 1024 * 1024phoenix/crates/serving/xai-recsys-engine/src/checkpoint_store.rs
Old lineNew lineFile
mode = "deluxe-4.5-internal"mode = "deluxe-4.6-internal"grox/flows/ptos/classifier.py
"learning_rate": 1e-3,"learning_rate": 7.1e-4,phoenix/xrex/configs/xrecsys.py
static_assert(sizeof(UpdateScalars) == 3 * sizeof(int32_t));static_assert(sizeof(UpdateScalars) == 4 * sizeof(int32_t));phoenix/xrex/cuda/async_emb/src/async_emb_kernel.hpp
input_output_aliases={12: 0, 13: 1, 14: 2},input_output_aliases={14: 0, 15: 1, 16: 2},phoenix/xrex/cutedsl/ranker_attention_varlen_fa4.py
input_output_aliases={12: 0},input_output_aliases={14: 0},phoenix/xrex/cutedsl/ranker_attention_varlen_fa4.py
prefault=False,prefault=True,phoenix/xrex/inference/model_runner.py
self._standby_emb_prefaulted = Falseself._standby_emb_prefaulted = Truephoenix/xrex/inference/model_runner.py

Schema fields (8)

Show proto/thrift field changes
ChangeField
addedFetchAggregatedUserActionRequests.feature_switch_overrides
addedNextActionDistribution.rewardOutputs
addedRewardOutputs.policyLogits
addedRewardOutputs.values
addedSlateContext.sidK1
addedSlateContext.sidK2
addedSlateContext.sidK3
addedSlateContext.sidKnown

Modules touched

ModuleFiles addedModifiedRemovedLines +Lines −
phoenix/2560+2,811−536
home-mixer/270+222−25
grox/160+146−88
visibility-filtering/140+124−2

Files (79)

Show file list
PathLinesNote
Mgrox/config/config.py+1/−0
Mgrox/flows/ptos/classifier.py+66/−7
Agrox/flows/ptos/prior_nsfw.py+34/−0
Mgrox/flows/ptos/state.py+1/−0
Mgrox/flows/ptos/task_safety_ptos_adult_content_cross_validation.py+2/−19
Mgrox/flows/ptos/task_safety_ptos_policy.py+40/−42
Mgrox/flows/ptos/task_safety_ptos_safemodel_sex_nudity.py+2/−20
Mhome-mixer/candidate_hydrators/conversation_gap_ancestor_hydrator.rs+22/−6
Mhome-mixer/candidate_hydrators/mod.rs+1/−0
Ahome-mixer/candidate_hydrators/quoted_post_text_hydrator.rs+106/−0
Mhome-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs+2/−2
Mhome-mixer/candidate_pipeline/reverse_chron_posts_pipeline.rs+7/−1
Ahome-mixer/filters/following_viewer_muted_keyword_filter.rs+65/−0
Mhome-mixer/filters/mod.rs+2/−1
Rhome-mixer/filters/viewer_muted_keyword_filter.rs+15/−15
Mhome-mixer/models/candidate.rs+2/−0
Mphoenix/Cargo.toml+2/−0
Mphoenix/crates/common/xai-recsys/src/util.rs+161/−21
Mphoenix/crates/serving/xai-recsys-engine/Cargo.toml+4/−1
Mphoenix/crates/serving/xai-recsys-engine/src/checkpoint_store.rs+175/−4
Mphoenix/crates/serving/xai-recsys-engine/src/python.rs+2/−52
Mphoenix/crates/serving/xai-recsys-engine/src/sid_client.rs+45/−0
Mphoenix/crates/serving/xai-recsys-engine/src/storage_util.rs+5/−4
Mphoenix/crates/serving/xai-recsys-engine/src/util.rs+121/−4
Mphoenix/crates/serving/xai-recsys-mm-server/src/mm_embedding_client.rs+13/−5
Mphoenix/crates/serving/xai-recsys-proto/proto/recsys.proto+26/−7
Mphoenix/python/common/xai-proto/hatch_build.py+8/−0
Mphoenix/python/common/xai-proto/proto/recsys.proto+26/−7
Mphoenix/xrex/configs/data_feeds.py+11/−0
Mphoenix/xrex/configs/xrecsys.py+108/−15
Mphoenix/xrex/configs/xrecsys_gen_recs.py+4/−4
Mphoenix/xrex/configs/xrecsys_sid_retrieval.py+4/−4
Mphoenix/xrex/configs/xrecsys_two_tower.py+4/−5
Mphoenix/xrex/cuda/async_emb/async_emb.py+64/−0
Mphoenix/xrex/cuda/async_emb/src/async_emb_api.cc+248/−35
Mphoenix/xrex/cuda/async_emb/src/async_emb_comm.cc+14/−1
Mphoenix/xrex/cuda/async_emb/src/async_emb_comm.hpp+5/−1
Mphoenix/xrex/cuda/async_emb/src/async_emb_kernel.cu+19/−2
Mphoenix/xrex/cuda/async_emb/src/async_emb_kernel.hpp+6/−1
Mphoenix/xrex/cutedsl/ranker_attention_varlen_fa4.py+193/−65
Mphoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py+37/−0
Mphoenix/xrex/cutedsl/ranker_fa4/flash_bwd_sm100.py+22/−2
Mphoenix/xrex/cutedsl/ranker_fa4/flash_fwd_sm100.py+12/−0
Mphoenix/xrex/cutedsl/ranker_fa4/mask.py+85/−2
Mphoenix/xrex/data/grpc_recsys.py+1/−0
Mphoenix/xrex/data/parquet_recsys.py+2/−0
Mphoenix/xrex/data/recsys/constants.py+16/−0
Mphoenix/xrex/data/recsys/feature_config.py+39/−12
Mphoenix/xrex/data/recsys/recsys_batch.py+47/−1
Mphoenix/xrex/data/retrieval_dataset.py+13/−9
Mphoenix/xrex/data/rust_kafka_recsys.py+27/−6
Mphoenix/xrex/data/rust_parquet_recsys.py+1/−0
Mphoenix/xrex/data/streaming/kafkaloader.py+126/−16
Mphoenix/xrex/eval/eval_utils.py+1/−2
Mphoenix/xrex/eval/metrics.py+1/−1
Mphoenix/xrex/inference/launch_inference.py+23/−7
Mphoenix/xrex/inference/model_runner.py+209/−158
Mphoenix/xrex/inference/pinned_d2h.py+13/−0
Mphoenix/xrex/inference/sid_retrieval_runner.py+10/−9
Mphoenix/xrex/models/recsys_attention.py+6/−0
Mphoenix/xrex/models/recsys_feature_prep.py+83/−2
Mphoenix/xrex/models/recsys_model.py+265/−25
Mphoenix/xrex/models/recsys_two_tower_model.py+11/−13
Mphoenix/xrex/optimizers/recsys/__init__.py+1/−0
Mphoenix/xrex/optimizers/recsys/async_emb_gradient_update.py+1/−1
Aphoenix/xrex/optimizers/recsys/dense_optim.py+47/−0
Aphoenix/xrex/optimizers/recsys/muon.py+240/−0
Mphoenix/xrex/optimizers/recsys/rowwise_adagrad.py+85/−4
Mphoenix/xrex/settings.py+1/−1
Mphoenix/xrex/train/misc.py+8/−0
Mphoenix/xrex/train/trainer.py+33/−16
Mphoenix/xrex/train/trainer_recsys.py+52/−4
Mphoenix/xrex/utils/log_timer.py+2/−1
Mphoenix/xrex/utils/metrics.py+23/−6
Mvisibility-filtering/clients/gizmoduck_client.rs+2/−2
Avisibility-filtering/dark_traffic_setup.rs+109/−0
Mvisibility-filtering/hydration/tes_hydrator.rs+8/−0
Mvisibility-filtering/lib.rs+1/−0
Mvisibility-filtering/main.rs+4/−0