Original query:

 Hash Join  (cost=685.59..128022.78 rows=769057 width=14) (actual time=57.415..12410.179 rows=984934 loops=1)
   Hash Cond: ((vzspll.zip_sample_point_id)::bpchar = zsp.id)
   CTE project_lat_longs
     ->  Unique  (cost=0.28..184.43 rows=4385 width=4) (actual time=0.024..23.244 rows=4385 loops=1)
           ->  Index Only Scan using index_addresses_on_latlong_id on addresses  (cost=0.28..168.98 rows=6180 width=4) (actual time=0.021..8.710 rows=6180 loops=1)
                 Heap Fetches: 0
   ->  Merge Join  (cost=353.39..103165.71 rows=1942695 width=14) (actual time=41.930..8321.370 rows=1868058 loops=1)
         Merge Cond: (pll.latlong_id = vzspll.lat_long_id)
         ->  Sort  (cost=352.96..363.92 rows=4385 width=4) (actual time=41.549..42.885 rows=616 loops=1)
               Sort Key: pll.latlong_id
               Sort Method: quicksort  Memory: 398kB
               ->  CTE Scan on project_lat_longs pll  (cost=0.00..87.70 rows=4385 width=4) (actual time=0.028..35.476 rows=4385 loops=1)
         ->  Index Scan using index_valid_zip_sample_point_lat_longs_on_lat_long_id on valid_zip_sample_point_lat_longs vzspll  (cost=0.44..77880.53 rows=1927189 width=14) (actual time=0.057..3136.473 ro
               Index Cond: ((lat_long_id >= 4500) AND (lat_long_id <= 5000))
   ->  Hash  (cost=96.23..96.23 rows=4123 width=10) (actual time=11.853..11.853 rows=4123 loops=1)
         Buckets: 8192  Batches: 1  Memory Usage: 234kB
         ->  Seq Scan on zip_sample_points zsp  (cost=0.00..96.23 rows=4123 width=10) (actual time=0.017..5.977 rows=4123 loops=1)
 Planning time: 1.180 ms
 Execution time: 13737.103 ms