Skip to content

Add materialize_cols to SparseLinOp and use it in the composite sketch - #275

Open
mmelnich wants to merge 1 commit into
mainfrom
cqrrto-followup/sparse-columns
Open

mmelnich wants to merge 1 commit into
mainfrom
cqrrto-followup/sparse-columns

Conversation

@mmelnich

Copy link
Copy Markdown
Contributor

Follow-up to the CQRRTO split of #129 (#237-#248, #255, #262).

Summary

Gives SparseLinOp the materialize_cols member (#241's HasMaterializeCols), which writes chosen columns of an operator into a buffer without a product. The code that reads an operator's columns when it can (the CholQR-family first Gram, the test-mode Q, the [A; mu I] sketch, materialize) now copies a sparse matrix's stored entries instead of multiplying it by identity columns. CompositeOperator's blocked sketch also copies its right operand's columns when that operand has the member.

Changes

  • linops/rl_sparse_linop.hh: materialize_cols for CSR, CSC and COO. It zeroes the requested columns and adds every stored entry in range (duplicates summed in storage order, unsorted indices fine), allocates nothing, and threads with OpenMP above kMaterializeColsParallelMin entries with results independent of the thread count. It checks range, ldb and shape like the other providers, accepts a COO window of a larger matrix (as the COO products do), and rejects a one-based matrix, as the products do (the views fix is SparseLinOp: keep the matrix's index base; check the sparse sketch #274).
  • linops/rl_composite_linop.hh: the blocked sketch fills each block from right_op.materialize_cols when available and then needs no identity block; its temporaries are now std::unique_ptr. The composite itself gets no materialize_cols, since its columns need a product.
  • linops/rl_materialize.hh: the SparseLinOp overload of materialize is removed; the overload for operators with the member makes the copies. For CSR and CSC, duplicate entries are now summed (the old overload kept the last) and a stored -0.0 reads as +0.0, both as the operator's own product does.
  • Numerics: for finite data without duplicates the copy is bit-identical to the product it replaces. Two duplicates sum the same either way; three or more can round differently. An Inf or NaN stays in its entry, where the product spread NaN along its row. In [A; mu I] with mu < 0, the identity block's zeros are now +0.0. Probes here found that MKL's CSC product loses one of two adjacent duplicates in a column; the copy sums them.
  • Benchmarks: in the scaling study the CholQR and sCholQR3 first passes copy instead of multiplying, and CholQR's modeled and measured memory drop the identity block. No other benchmark output changes on generated matrices or on Matrix Market files without repeated coordinates.
  • RandLAPACK::testing::ProductsOnly (forwards an operator's products and hides materialize_cols) moves into testing/rl_test_utils.hh, replacing a copy in the extras tests.
  • Tests: TestSparseMaterializeCols (every format, duplicates, unsorted indices, windows, a threaded case, special values, argument checks), a composite sketch test that counts copies and products, a [SparseLinOp; mu I] sketch test, and the four CholQR-family drivers and CQRRTO on sparse operators against the product path. RandLAPACK_tests 742/742, extras 176/176, benchmark tree builds.

SparseLinOp now provides the optional materialize_cols hook
(HasMaterializeCols) for CSR, CSC and COO, with the overflow-safe range
check and the ldb >= n_rows check of the other providers. A zero count,
or an operator with no rows, returns; nothing is allocated. Rows
0..n_rows-1 of the requested columns are zeroed and every stored entry
in range is added, so duplicates are summed. CSC reads the requested
columns directly; CSR scans every row in full (its column indices need
not be sorted) and COO every entry. As in the sparse-dense products, a
CSR or CSC operator must have its matrix's dimensions, a COO operator
reads the leading n_rows x n_cols window of a larger matrix, and a
one-based matrix is rejected (the sketch with a sparse S needs equal
dimensions; see #274). Loops that touch at least
kMaterializeColsParallelMin = 2^15 entries run with OpenMP: the zero
fill over columns, CSC's fill and scatter over columns (counting the
entries of the requested columns), CSR's scatter over rows (counting
all its entries); COO's scatter stays serial. Each output entry is
written by one thread, duplicates in storage order, so the result does
not depend on the thread count.

CompositeOperator's blocked sketch (block_size > 0, ColMajor,
Side::Right, NoTrans) copies the right operand's columns through
materialize_cols when the operand provides it (DenseLinOp and
SparseLinOp are the tested cases), instead of applying it to identity
columns, and then allocates no identity block. The composite's other
requirements on its operands, their Side-taking and sketch products,
still apply. CompositeOperator itself gets no materialize_cols: its
columns need a product. The sketch operator's temporary buffers are now
std::unique_ptr, so a throwing operand no longer leaks them.

materialize's SparseLinOp overload (zero fill, then sparse_to_dense) is
removed; a SparseLinOp takes the overload for operators with the hook.
COO output is unchanged (coo_to_dense also adds in storage order onto
zero). CSR and CSC output changes where sparse_to_dense disagreed with
the operator's product: it kept the last of duplicate entries (now
summed) and a stored -0.0 as -0.0 (now +0.0). The m x n temporary it
allocated when ldb > m is gone.

Numerics where a SparseLinOp's columns are now copied instead of
multiplied by identity columns (the unpreconditioned first Gram of
CholQR, CholQR2 and sCholQR3; the first product of
sCholQR3_linops_basic; the test-mode Q of the CholQR family and of
CQRRTO_linops; the sketch, first Gram, test-mode Q and materialize of
[A; mu I]; the benchmark's KW reference; the composite sketch): for
finite data with no duplicate entries each entry of such a product is
one stored value plus exact zeros, so the copied sparse block is
bit-identical to it, and the rest of each computation is unchanged.
Both read a stored -0.0 and an empty entry as +0.0 (checked on MKL and
on RandBLAS's kernels). The copy sums duplicates in storage order and
the product in the backend's order, so two duplicates give the same sum
and three or more can round differently. MKL's CSC product
(mkl_sparse_d_mm, transposed, on the CSR of A^T) lost one of two
duplicates stored next to each other in a column in probes here, where
the copy, RandBLAS's CSC kernel and MKL's CSR and COO products sum
them. An Inf or NaN stays in its own entry in the copy; the product
turns the rest of its row in the block into NaN (Inf * 0). Subnormals
are exact unless the floating-point environment flushes them (FTZ/DAZ),
which affects the copy's additions and the product alike. In [A; mu I]
with mu < 0, the mu I block's zeros were -0.0 from the product
(mu * 0) and are +0.0 in the copy. A DenseLinOp right operand of the
composite sketch has the numerics of DenseLinOp's hook.

In the Q-less scaling study (CQRRTO_linop_basic) the CholQR and sCholQR3
rows copy the CSR operator's columns in their first Gram, so CholQR's
fwd and sCholQR3's fwd1 slots time the copy, and CholQR's model
(cholqr_linops_analytical_kb, through HasMaterializeCols) and peak drop
the n x b identity block. Its check of Q and the CQRRT_expl row
materialize through the hook: values are unchanged on its generated
matrices and on Matrix Market files without repeated coordinates, and a
file with repeated coordinates now gets their sum, as the operator's
product does. CQRRTO_linops on a bare SparseLinOp changes only in its
test-mode Q, and Blendenpik_linops on one not at all. No other
benchmark output changes.

test_helpers' ProductsOnly moves to RandLAPACK/testing/rl_test_utils.hh
as RandLAPACK::testing::ProductsOnly, which forwards every product the
operator accepts (requires std::invocable) and hides materialize_cols;
the extras' Toeplitz tests use it instead of their own. The docstring
of sparse_to_dense_summing_duplicates now says only COO sums
duplicates. rl_materialize.hh drops includes it no longer needs.

Tests: materialize_cols in each format, in double and float, on a matrix
with a triple and a double duplicate, unsorted CSR and CSC indices and
an empty row and column, at ranges at the start, middle and end and with
zero counts, with padding rows untouched, equal to the product with
identity columns there for CSR and COO and on duplicate-free random data
for all three; a COO window of a larger matrix; a 16384 x 10 copy sized
from kMaterializeColsParallelMin so that every loop is threaded, on at
least 4 OpenMP threads; Inf, NaN and -0.0; materialize summing
duplicates; the argument and shape checks; the one-based guard, on a
view marked one-based by hand; an operator with no rows and a matrix
with no stored entries. The composite sketch with a DenseLinOp and a
SparseLinOp right operand counts that the copy path reads n columns and
never applies the operand, and the product path applies it once per
block. The sketch of [SparseLinOp; mu I], the CholQR family on a CSR
SparseLinOp and on [SparseLinOp; mu I], CholQR_linops on a COO window
and CQRRTO_linops on the stack match the same operators behind
ProductsOnly. static_asserts check that ProductsOnly accepts exactly its
operator's calls, and an explicit instantiation that TransposedLinOp
still detects a missing Side-taking product through it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant