x-algorithm changelog (unofficial)

Unofficial, community-maintained project. Not affiliated with, endorsed by, or operated by X Corp. or xAI. Notes are generated from the public git history of xai-org/x-algorithm and may contain mistakes; the upstream diff is the source of truth.

· e414c17 · 187 files, +18,263/−926

May 15 release: ads blending, Grox, more sources, hydrators and filters

ads blendersartifactscandidate hydratorscandidate sourcesfiltersnumeric tweaksscorersselectorsside effects

A large update (187 files, +18,263/−926). The upstream README gained an "Updates — May 15th, 2026" section for this release, which is no longer in the current README.

  • New candidate sources in home-mixer/sources/: ads_source, cached_posts_source, phoenix_moe_source, phoenix_topics_source, prompts_source, push_to_home_source, scored_posts_source, tweet_mixer_source and who_to_follow_source.
  • New scorers: ranking_scorer and vm_ranker (the original weighted_scorer, oon_scorer and author_diversity_scorer files are still present).
  • New filters: ancillary_vf_filter, new_user_topic_ids_filter, previously_seen_posts_backup_filter, topic_ids_filter and video_filter.
  • New selector: blender_selector, plus a new home-mixer/ads/ module for placing ads in the feed.
  • Adds the grox/ content-understanding service (classifiers, embedders, task engine) and many new query/candidate hydrators.
  • Phoenix gets a single run_pipeline.py entry point and a pre-trained mini model distributed through Git LFS (phoenix/artifacts/oss-phoenix-artifacts.zip, about 3.1 GB).

Full analysis → Summary: hand-written notes, checked against the diff

Automatic analysis

Config parameters (param! defaults)

None.

Pipeline components

KindChangeFile
Ads blenderaddedhome-mixer/ads/partition_organic_blender.rs
Ads blenderaddedhome-mixer/ads/safe_gap_blender.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/ads_brand_safety_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/ads_brand_safety_vf_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/blocked_by_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/engagement_counts_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/filtered_topics_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/following_replied_users_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/has_media_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/language_code_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/mutual_follow_jaccard_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/quote_hydrator.rs
Candidate hydratoraddedhome-mixer/candidate_hydrators/tweet_type_metrics_hydrator.rs
Filteraddedhome-mixer/filters/ancillary_vf_filter.rs
Filteraddedhome-mixer/filters/new_user_topic_ids_filter.rs
Filteraddedhome-mixer/filters/previously_seen_posts_backup_filter.rs
Filteraddedhome-mixer/filters/topic_ids_filter.rs
Filteraddedhome-mixer/filters/video_filter.rs
Candidate hydratoraddedhome-mixer/query_hydrators/blocked_user_ids_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/cached_posts_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/followed_grok_topics_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/followed_starter_packs_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/followed_user_ids_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/impressed_posts_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/impression_bloom_filter_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/inferred_grok_topics_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/ip_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/muted_user_ids_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/mutual_follow_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/past_request_timestamps_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/retrieval_sequence_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/scoring_sequence_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/served_history_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/subscribed_user_ids_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/user_demographics_query_hydrator.rs
Candidate hydratoraddedhome-mixer/query_hydrators/user_inferred_gender_query_hydrator.rs
Scoreraddedhome-mixer/scorers/ranking_scorer.rs
Scoreraddedhome-mixer/scorers/vm_ranker.rs
Selectoraddedhome-mixer/selectors/blender_selector.rs
Side effectaddedhome-mixer/side_effects/ads_injection_logging_side_effect.rs
Side effectaddedhome-mixer/side_effects/client_events_kafka_side_effect.rs
Side effectaddedhome-mixer/side_effects/for_you_response_stats_side_effect.rs
Side effectaddedhome-mixer/side_effects/mutual_follow_stats_side_effect.rs
Side effectaddedhome-mixer/side_effects/phoenix_experiments_side_effect.rs
Side effectaddedhome-mixer/side_effects/phoenix_request_cache_side_effect.rs
Side effectaddedhome-mixer/side_effects/publish_seen_ids_to_kafka_side_effect.rs
Side effectaddedhome-mixer/side_effects/redis_post_candidate_cache_side_effect.rs
Side effectaddedhome-mixer/side_effects/reranking_kafka_side_effect.rs
Side effectaddedhome-mixer/side_effects/scored_stats_side_effect.rs
Side effectaddedhome-mixer/side_effects/served_candidates_kafka_side_effect.rs
Side effectaddedhome-mixer/side_effects/truncate_served_history_side_effect.rs
Side effectaddedhome-mixer/side_effects/update_past_request_timestamps_side_effect.rs
Side effectaddedhome-mixer/side_effects/update_served_history_side_effect.rs
Candidate sourceaddedhome-mixer/sources/ads_source.rs
Candidate sourceaddedhome-mixer/sources/cached_posts_source.rs
Candidate sourceaddedhome-mixer/sources/phoenix_moe_source.rs
Candidate sourceaddedhome-mixer/sources/phoenix_topics_source.rs
Candidate sourceaddedhome-mixer/sources/prompts_source.rs
Candidate sourceaddedhome-mixer/sources/push_to_home_source.rs
Candidate sourceaddedhome-mixer/sources/scored_posts_source.rs
Candidate sourceaddedhome-mixer/sources/tweet_mixer_source.rs
Candidate sourceaddedhome-mixer/sources/who_to_follow_source.rs
Candidate hydratormodifiedcandidate-pipeline/query_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/core_data_candidate_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/gizmoduck_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/in_network_candidate_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/subscription_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/vf_candidate_hydrator.rs
Candidate hydratormodifiedhome-mixer/candidate_hydrators/video_duration_candidate_hydrator.rs
Filtermodifiedhome-mixer/filters/age_filter.rs
Filtermodifiedhome-mixer/filters/author_socialgraph_filter.rs
Filtermodifiedhome-mixer/filters/core_data_hydration_filter.rs
Filtermodifiedhome-mixer/filters/dedup_conversation_filter.rs
Filtermodifiedhome-mixer/filters/drop_duplicates_filter.rs
Filtermodifiedhome-mixer/filters/ineligible_subscription_filter.rs
Filtermodifiedhome-mixer/filters/muted_keyword_filter.rs
Filtermodifiedhome-mixer/filters/previously_seen_posts_filter.rs
Filtermodifiedhome-mixer/filters/previously_served_posts_filter.rs
Filtermodifiedhome-mixer/filters/retweet_deduplication_filter.rs
Filtermodifiedhome-mixer/filters/self_tweet_filter.rs
Filtermodifiedhome-mixer/filters/vf_filter.rs
Scorermodifiedhome-mixer/scorers/phoenix_scorer.rs
Selectormodifiedhome-mixer/selectors/top_k_score_selector.rs
Candidate sourcemodifiedhome-mixer/sources/phoenix_source.rs
Candidate sourcemodifiedhome-mixer/sources/thunder_source.rs

Changed constants & literal values

Old lineNew lineFile
num_user_embeddings: int = 100000,num_user_embeddings: int = 1_000_000,phoenix/runners.py
num_post_embeddings: int = 100000,num_post_embeddings: int = 1_000_000,phoenix/runners.py
num_author_embeddings: int = 100000,num_author_embeddings: int = 1_000_000,phoenix/runners.py

Modules touched

ModuleFiles addedModifiedRemovedLines +Lines −
home-mixer/74340+10,052−671
grox/5900+6,500−0
phoenix/270+1,039−66
candidate-pipeline/180+649−189
(repo root)/110+23−0

Files (187)

Show file list
PathLinesNote
A.gitattributes+2/−0
MREADME.md+21/−0
Mcandidate-pipeline/candidate_pipeline.rs+313/−149
Mcandidate-pipeline/filter.rs+45/−7
Mcandidate-pipeline/hydrator.rs+157/−7
Mcandidate-pipeline/query_hydrator.rs+15/−1
Mcandidate-pipeline/scorer.rs+34/−8
Mcandidate-pipeline/selector.rs+51/−11
Mcandidate-pipeline/side_effect.rs+11/−3
Mcandidate-pipeline/source.rs+20/−3
Acandidate-pipeline/util.rs+3/−0
Agrox/__init__.py+0/−0
Agrox/classifiers/content/banger_initial_screen.py+160/−0
Agrox/classifiers/content/classifier.py+98/−0
Agrox/classifiers/content/post_safety_screen_deluxe.py+92/−0
Agrox/classifiers/content/reply_ranking.py+158/−0
Agrox/classifiers/content/safety_ptos.py+288/−0
Agrox/classifiers/content/spam.py+104/−0
Agrox/data_loaders/asr_processor.py+394/−0
Agrox/data_loaders/kafka_loader.py+232/−0
Agrox/data_loaders/message_queue_loader.py+39/−0
Agrox/data_loaders/strato_loader.py+154/−0
Agrox/dispatcher.py+370/−0
Agrox/embedder/multimodal_post_embedder_v2.py+287/−0
Agrox/embedder/multimodal_post_embedder_v5.py+120/−0
Agrox/engine.py+137/−0
Agrox/generators/stream_generator.py+222/−0
Agrox/generators/task_generator.py+153/−0
Agrox/lib/stream.py+46/−0
Agrox/lib/utils.py+6/−0
Agrox/main.py+54/−0
Agrox/plans/plan.py+106/−0
Agrox/plans/plan_initial_banger.py+37/−0
Agrox/plans/plan_master.py+62/−0
Agrox/plans/plan_post_embedding_v5.py+29/−0
Agrox/plans/plan_post_embedding_v5_for_reply.py+32/−0
Agrox/plans/plan_post_embedding_with_summary.py+38/−0
Agrox/plans/plan_post_embedding_with_summary_for_reply.py+38/−0
Agrox/plans/plan_post_safety.py+32/−0
Agrox/plans/plan_reply_ranking.py+27/−0
Agrox/plans/plan_safety_ptos.py+32/−0
Agrox/plans/plan_spam_comment.py+29/−0
Agrox/schedules/context.py+47/−0
Agrox/schedules/init.py+35/−0
Agrox/schedules/types.py+73/−0
Agrox/summarizer/eapi_summarizer.py+44/−0
Agrox/summarizer/post_embedding_summarizer.py+55/−0
Agrox/summarizer/summarizer.py+44/−0
Agrox/tasks/disable_rules.py+56/−0
Agrox/tasks/task.py+150/−0
Agrox/tasks/task_asr.py+74/−0
Agrox/tasks/task_banger_screen.py+79/−0
Agrox/tasks/task_embedding_pub.py+79/−0
Agrox/tasks/task_filters.py+370/−0
Agrox/tasks/task_grok_upa_action_with_labels.py+57/−0
Agrox/tasks/task_load_post_with_not_found_retry.py+35/−0
Agrox/tasks/task_load_post_with_summary.py+27/−0
Agrox/tasks/task_load_user_recent_posts.py+23/−0
Agrox/tasks/task_media.py+21/−0
Agrox/tasks/task_multimodal_post_embedding.py+86/−0
Agrox/tasks/task_post_safety_screen_deluxe.py+25/−0
Agrox/tasks/task_pub.py+555/−0
Agrox/tasks/task_rank_replies.py+31/−0
Agrox/tasks/task_rate_limit.py+191/−0
Agrox/tasks/task_safety_ptos_category.py+61/−0
Agrox/tasks/task_safety_ptos_policy.py+89/−0
Agrox/tasks/task_spam_detection.py+57/−0
Agrox/tasks/task_summarizer_for_post_embedding.py+25/−0
Agrox/tasks/task_write_mm_embedding_sink.py+193/−0
Agrox/tasks/task_write_safety_post_annotations_result_sink.py+342/−0
Ahome-mixer/ads/mod.rs+20/−0
Ahome-mixer/ads/partition_organic_blender.rs+190/−0
Ahome-mixer/ads/safe_gap_blender.rs+95/−0
Ahome-mixer/ads/util.rs+228/−0
Ahome-mixer/candidate_hydrators/ads_brand_safety_hydrator.rs+176/−0
Ahome-mixer/candidate_hydrators/ads_brand_safety_vf_hydrator.rs+108/−0
Ahome-mixer/candidate_hydrators/blocked_by_hydrator.rs+57/−0
Mhome-mixer/candidate_hydrators/core_data_candidate_hydrator.rs+97/−22
Ahome-mixer/candidate_hydrators/engagement_counts_hydrator.rs+113/−0
Ahome-mixer/candidate_hydrators/filtered_topics_hydrator.rs+134/−0
Ahome-mixer/candidate_hydrators/following_replied_users_hydrator.rs+98/−0
Mhome-mixer/candidate_hydrators/gizmoduck_hydrator.rs+97/−31
Ahome-mixer/candidate_hydrators/has_media_hydrator.rs+85/−0
Mhome-mixer/candidate_hydrators/in_network_candidate_hydrator.rs+12/−11
Ahome-mixer/candidate_hydrators/language_code_hydrator.rs+84/−0
Mhome-mixer/candidate_hydrators/mod.rs+10/−0
Ahome-mixer/candidate_hydrators/mutual_follow_jaccard_hydrator.rs+118/−0
Ahome-mixer/candidate_hydrators/quote_hydrator.rs+176/−0
Mhome-mixer/candidate_hydrators/subscription_hydrator.rs+47/−15
Ahome-mixer/candidate_hydrators/tweet_type_metrics_hydrator.rs+180/−0
Mhome-mixer/candidate_hydrators/vf_candidate_hydrator.rs+82/−21
Mhome-mixer/candidate_hydrators/video_duration_candidate_hydrator.rs+53/−30
Ahome-mixer/candidate_pipeline/for_you_candidate_pipeline.rs+278/−0
Mhome-mixer/candidate_pipeline/mod.rs+1/−4
Mhome-mixer/candidate_pipeline/phoenix_candidate_pipeline.rs+605/−88
Mhome-mixer/filters/age_filter.rs+8/−10
Ahome-mixer/filters/ancillary_vf_filter.rs+19/−0
Mhome-mixer/filters/author_socialgraph_filter.rs+36/−17
Mhome-mixer/filters/core_data_hydration_filter.rs+6/−10
Mhome-mixer/filters/dedup_conversation_filter.rs+6/−9
Mhome-mixer/filters/drop_duplicates_filter.rs+5/−7
Mhome-mixer/filters/ineligible_subscription_filter.rs+5/−7
Mhome-mixer/filters/mod.rs+5/−0
Mhome-mixer/filters/muted_keyword_filter.rs+25/−27
Ahome-mixer/filters/new_user_topic_ids_filter.rs+31/−0
Ahome-mixer/filters/previously_seen_posts_backup_filter.rs+29/−0
Mhome-mixer/filters/previously_seen_posts_filter.rs+7/−9
Mhome-mixer/filters/previously_served_posts_filter.rs+13/−8
Mhome-mixer/filters/retweet_deduplication_filter.rs+10/−21
Mhome-mixer/filters/self_tweet_filter.rs+6/−8
Ahome-mixer/filters/topic_ids_filter.rs+571/−0
Mhome-mixer/filters/vf_filter.rs+5/−8
Ahome-mixer/filters/video_filter.rs+23/−0
Ahome-mixer/for_you_server.rs+87/−0
Mhome-mixer/lib.rs+9/−6
Mhome-mixer/main.rs+48/−63
Ahome-mixer/models/brand_safety.rs+91/−0
Ahome-mixer/models/candidate.rs+153/−0
Ahome-mixer/models/candidate_features.rs+146/−0
Ahome-mixer/models/in_network_reply.rs+23/−0
Ahome-mixer/models/mod.rs+7/−0
Ahome-mixer/models/query.rs+248/−0
Ahome-mixer/models/user_features.rs+80/−0
Ahome-mixer/query_hydrators/blocked_user_ids_query_hydrator.rs+33/−0
Ahome-mixer/query_hydrators/cached_posts_query_hydrator.rs+82/−0
Ahome-mixer/query_hydrators/followed_grok_topics_query_hydrator.rs+75/−0
Ahome-mixer/query_hydrators/followed_starter_packs_query_hydrator.rs+44/−0
Ahome-mixer/query_hydrators/followed_user_ids_query_hydrator.rs+33/−0
Ahome-mixer/query_hydrators/impressed_posts_query_hydrator.rs+25/−0
Ahome-mixer/query_hydrators/impression_bloom_filter_query_hydrator.rs+50/−0
Ahome-mixer/query_hydrators/inferred_grok_topics_query_hydrator.rs+39/−0
Ahome-mixer/query_hydrators/ip_query_hydrator.rs+30/−0
Mhome-mixer/query_hydrators/mod.rs+18/−2
Ahome-mixer/query_hydrators/muted_user_ids_query_hydrator.rs+33/−0
Ahome-mixer/query_hydrators/mutual_follow_query_hydrator.rs+37/−0
Ahome-mixer/query_hydrators/past_request_timestamps_query_hydrator.rs+36/−0
Ahome-mixer/query_hydrators/retrieval_sequence_query_hydrator.rs+71/−0
Ahome-mixer/query_hydrators/scoring_sequence_query_hydrator.rs+68/−0
Ahome-mixer/query_hydrators/served_history_query_hydrator.rs+109/−0
Ahome-mixer/query_hydrators/subscribed_user_ids_query_hydrator.rs+33/−0
Ahome-mixer/query_hydrators/user_demographics_query_hydrator.rs+30/−0
Ahome-mixer/query_hydrators/user_inferred_gender_query_hydrator.rs+58/−0
Ahome-mixer/scored_posts_server.rs+215/−0
Mhome-mixer/scorers/mod.rs+2/−3
Mhome-mixer/scorers/phoenix_scorer.rs+95/−147
Ahome-mixer/scorers/ranking_scorer.rs+290/−0
Ahome-mixer/scorers/vm_ranker.rs+132/−0
Ahome-mixer/selectors/blender_selector.rs+164/−0
Mhome-mixer/selectors/mod.rs+2/−0
Mhome-mixer/selectors/top_k_score_selector.rs+2/−2
Mhome-mixer/server.rs+394/−56
Ahome-mixer/side_effects/ads_injection_logging_side_effect.rs+146/−0
Ahome-mixer/side_effects/client_events_kafka_side_effect.rs+302/−0
Ahome-mixer/side_effects/for_you_response_stats_side_effect.rs+157/−0
Mhome-mixer/side_effects/mod.rs+14/−1
Ahome-mixer/side_effects/mutual_follow_stats_side_effect.rs+67/−0
Ahome-mixer/side_effects/phoenix_experiments_side_effect.rs+174/−0
Ahome-mixer/side_effects/phoenix_request_cache_side_effect.rs+131/−0
Ahome-mixer/side_effects/publish_seen_ids_to_kafka_side_effect.rs+79/−0
Ahome-mixer/side_effects/redis_post_candidate_cache_side_effect.rs+92/−0
Ahome-mixer/side_effects/reranking_kafka_side_effect.rs+157/−0
Ahome-mixer/side_effects/scored_stats_side_effect.rs+281/−0
Ahome-mixer/side_effects/served_candidates_kafka_side_effect.rs+207/−0
Ahome-mixer/side_effects/truncate_served_history_side_effect.rs+53/−0
Ahome-mixer/side_effects/update_past_request_timestamps_side_effect.rs+61/−0
Ahome-mixer/side_effects/update_served_history_side_effect.rs+268/−0
Ahome-mixer/sources/ads_source.rs+60/−0
Ahome-mixer/sources/cached_posts_source.rs+17/−0
Mhome-mixer/sources/mod.rs+9/−0
Ahome-mixer/sources/phoenix_moe_source.rs+73/−0
Mhome-mixer/sources/phoenix_source.rs+77/−12
Ahome-mixer/sources/phoenix_topics_source.rs+106/−0
Ahome-mixer/sources/prompts_source.rs+98/−0
Ahome-mixer/sources/push_to_home_source.rs+97/−0
Ahome-mixer/sources/scored_posts_source.rs+32/−0
Mhome-mixer/sources/thunder_source.rs+44/−16
Ahome-mixer/sources/tweet_mixer_source.rs+103/−0
Ahome-mixer/sources/who_to_follow_source.rs+101/−0
Mphoenix/README.md+72/−5
Aphoenix/artifacts/oss-phoenix-artifacts.zip+3/−0
Mphoenix/grok.py+35/−5
Mphoenix/recsys_model.py+230/−24
Mphoenix/recsys_retrieval_model.py+17/−1
Aphoenix/run_pipeline.py+393/−0
Mphoenix/runners.py+81/−3
Mphoenix/test_recsys_model.py+147/−25
Mphoenix/test_recsys_retrieval_model.py+61/−3