x-algorithm changelog (unofficial)

Unofficial, community-maintained project. Not affiliated with, endorsed by, or operated by X Corp. or xAI. Notes are generated from the public git history of xai-org/x-algorithm and may contain mistakes; the upstream diff is the source of truth.

· 76843a5 · 105 files, +5,227/−756

New semantic-ID candidate source (off by default)

candidate sourcesconstantsnumeric tweaksparamsschema
  • New candidate source SidSource (home-mixer/sources/sid_source.rs) that retrieves posts through a new SidRetrievalClient (seeded with up to SidSourceMaxSeeds post IDs taken from the viewer's signals via post_signal_ids). It is gated by EnableSidSource (default off) with SidSourceMaxResults = 800, SidSourceMaxSeeds = 50, SidSourceMaxPerSeed = 100 and SidSourceMinPrefixDepth = 3.
  • Removes the TestBoost and TestCap params.
  • Many proto/schema additions and Phoenix changes in the same sync. See the full analysis.

Full analysis → Summary: hand-written notes, checked against the diff

Automatic analysis

Config parameters (param! defaults)

Added parameterTypeDefaultFile
EnableSidSourceboolfalsehome-mixer/params/param.rs
SidSourceMaxPerSeedu32100home-mixer/params/param.rs
SidSourceMaxResultsu32800home-mixer/params/param.rs
SidSourceMaxSeedsusize50home-mixer/params/param.rs
SidSourceMinPrefixDepthu323home-mixer/params/param.rs
Removed parameterTypeLast defaultNote
TestBoostf641.0removed from home-mixer/util/author_rules.rs
TestCapf6410.0removed from home-mixer/util/author_rules.rs

Pipeline components

KindChangeFile
Candidate sourceaddedhome-mixer/sources/sid_source.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/engagement_counts_hydrator.rs
Scorermodifiedhome-mixer/scorers/author_cold_start.rs
Candidate sourcemodifiedhome-mixer/sources/phoenix_moe_source.rs
Candidate sourcemodifiedhome-mixer/sources/simclusters_source.rs

Changed constants & literal values

NameOldNewFile
LATENCY_MS"safety_labels_lookup_latency_ms""safety_labels_lookup_latency_ms_vm"visibility-filtering/safety_label_source/metrics.rs
Old lineNew lineFile
cold_start_candidate(1, minutes(10), 3),cold_start_candidate(1, minutes(10), 1000),home-mixer/scorers/author_cold_start.rs

Schema fields (9)

Show proto/thrift field changes
ChangeField
addedAdInfo.viewer_advertiser_conversion_counts
addedAdInfo.viewer_advertiser_engagement_counts
addedEngagementCounts.collected_actions
addedExcludedPostsBloomFilter.bit_array
addedExcludedPostsBloomFilter.false_positive_rate
addedExcludedPostsBloomFilter.size_cap
addedRetrieveTopKCandidatesRequest.excluded_post_ids
addedRetrieveTopKCandidatesRequest.excluded_posts_bloom_filters
addedRetrieveTopKCandidatesRequest.seed_post_ids

Modules touched

ModuleFiles addedModifiedRemovedLines +Lines −
phoenix/13110+4,196−222
home-mixer/2101+446−252
visibility-filtering/0200+346−81
visibility-filtering-client/130+79−47
abuse-enforcement-service/070+57−57
botmaker/0290+56−56
thunder/020+21−21
phoenix-rankall/020+15−15
media-model-proxy/020+3−3
under-the-hood/010+6−0
vm-ranker/010+2−2

Files (105)

Show file list
PathLinesNote
Mabuse-enforcement-service/service-lib/rules/enforcement_post.yaml+3/−3formatting only
Mabuse-enforcement-service/service-lib/rules/enforcement_user.yaml+7/−7formatting only
Mabuse-enforcement-service/service-lib/src/lib.rs+9/−9formatting only
Mabuse-enforcement-service/service-lib/src/metrics.rs+9/−9formatting only
Mabuse-enforcement-service/service-lib/src/overturn_hold.rs+22/−22formatting only
Mabuse-enforcement-service/service-lib/src/rules.rs+4/−4formatting only
Mabuse-enforcement-service/service-lib/src/service.rs+3/−3formatting only
Mbotmaker/scarecrow_features/src/main/java/com/twitter/scarecrow/features/FeaturesOfMediaUpdate.java+1/−1formatting only
Mbotmaker/scarecrow_features/src/main/java/com/twitter/scarecrow/features/FeaturesOfTweet.java+2/−2formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/Compiler.java+2/−2formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/Parser.java+4/−4formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/RegexCache.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/Serializer.java+2/−2formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/thrift/TFieldRequirementType.java+3/−3formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/thrift/TType.java+14/−14formatting only
Mbotmaker/src/java/com/twitter/botmaker/compiler/types/Type.java+2/−2formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/Constant.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/Variable.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/Any.java+2/−2formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/Filter.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/ForEach.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/ListOperator.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/MapEntries.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/MapMerge.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/MapRemove.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/PairSecond.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/SetDiff.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/collection/SetSymmetricDiff.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/conversion/ToMap.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/conversion/ToPair.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/feature/Exists.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/logic/BatchedOr.java+3/−3formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/misc/TestRun.java+3/−3formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/string/HammingDist.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/thrift/ThriftOperator.java+1/−1formatting only
Mbotmaker/src/java/com/twitter/botmaker/function/thrift/ThriftStructOperator.java+1/−1formatting only
Mhome-mixer/candidate_hydrators/engagement_counts_hydrator.rs+16/−48
Mhome-mixer/candidate_hydrators/tweet_type_metrics_hydrator.rs+5/−5formatting only
Mhome-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs+13/−1
Mhome-mixer/clients/mod.rs+1/−0
Ahome-mixer/clients/sid_retrieval_client.rs+246/−0
Mhome-mixer/params/param.rs+31/−1
Mhome-mixer/scorers/author_cold_start.rs+15/−38
Mhome-mixer/sources/mod.rs+1/−0
Mhome-mixer/sources/phoenix_moe_source.rs+4/−0
Ahome-mixer/sources/sid_source.rs+112/−0
Mhome-mixer/sources/simclusters_source.rs+2/−2
Dhome-mixer/util/author_rules.rs+0/−156
Mhome-mixer/util/mod.rs+0/−1
Mmedia-model-proxy/integrationtest/scripts/workflow.sh+1/−1formatting only
Mmedia-model-proxy/loadtest/config/loadtest.yaml+2/−2formatting only
Mphoenix-rankall/src/pipeline/mod.rs+3/−3formatting only
Mphoenix-rankall/src/store/base.rs+12/−12formatting only
Mphoenix/crates/serving/xai-recsys-proto/proto/recsys.proto+68/−44
Mphoenix/python/common/xai-proto/proto/recsys.proto+68/−44
Aphoenix/python/training/xai-checkpointing/xai_checkpointing/kms_client.py+74/−0
Mphoenix/python/training/xai-checkpointing/xai_checkpointing/load.py+2/−3
Aphoenix/xrex/cuda/row_emb/__init__.py+2/−0
Aphoenix/xrex/cuda/row_emb/comm_utils.py+47/−0
Aphoenix/xrex/cuda/row_emb/row_emb.py+290/−0
Aphoenix/xrex/cuda/row_emb/src/comm_utils.cc+146/−0
Aphoenix/xrex/cuda/row_emb/src/comm_utils.h+57/−0
Aphoenix/xrex/cuda/row_emb/src/cuda_error_utils.hpp+68/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_api.cc+766/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_comm.cc+1104/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_comm.hpp+256/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_common.cuh+82/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_kernel.cu+629/−0
Aphoenix/xrex/cuda/row_emb/src/row_emb_kernel.hpp+135/−0
Mphoenix/xrex/cutedsl/ranker_attention_varlen_fa4.py+36/−6
Mphoenix/xrex/data/recsys/sequence_packing.py+2/−0
Mphoenix/xrex/data/rust_kafka_recsys.py+112/−36
Mphoenix/xrex/data/streaming/kafkaloader.py+7/−4
Mphoenix/xrex/inference/model_runner.py+8/−15
Mphoenix/xrex/optimizers/recsys/rowwise_adagrad.py+14/−9
Mphoenix/xrex/train/trainer_recsys.py+164/−39
Mphoenix/xrex/utils/recsys_async_emb.py+59/−22
Mthunder/kafka/tweet_events_listener.rs+1/−1formatting only
Mthunder/posts/post_store.rs+20/−20formatting only
Munder-the-hood/strato/lib/underTheHoodLabels.strato+6/−0
Mvisibility-filtering-client/lib.rs+2/−0
Avisibility-filtering-client/test_support.rs+37/−0
Mvisibility-filtering-client/tweet_safety_label.rs+39/−12
Mvisibility-filtering-client/vf_client.rs+1/−35
Mvisibility-filtering/caller_identity.rs+90/−15
Mvisibility-filtering/evaluate_tweets.rs+3/−1
Mvisibility-filtering/filter.rs+94/−2
Mvisibility-filtering/filter_tweets.rs+2/−1
Mvisibility-filtering/get_safety_labels.rs+3/−1
Mvisibility-filtering/hydration/execute.rs+68/−2
Mvisibility-filtering/hydration/plan.rs+9/−5
Mvisibility-filtering/hydration/store.rs+12/−4
Mvisibility-filtering/hydration/tweet_source.rs+8/−14
Mvisibility-filtering/models/mod.rs+1/−0
Mvisibility-filtering/models/tweet.rs+0/−5
Mvisibility-filtering/retweet.rs+5/−6
Mvisibility-filtering/rules/context.rs+5/−0
Mvisibility-filtering/rules/fixtures.rs+1/−1
Mvisibility-filtering/rules/golden_corpus/age_gating.rs+7/−5
Mvisibility-filtering/rules/golden_corpus/exclusive_content.rs+1/−5
Mvisibility-filtering/rules/golden_corpus/oon_media.rs+5/−5
Mvisibility-filtering/rules/golden_corpus/tweet_state.rs+1/−1
Mvisibility-filtering/rules/rule_spec.rs+2/−1
Mvisibility-filtering/safety_label_source/metrics.rs+29/−7
Mvm-ranker/metrics.rs+2/−2formatting only