Repository navigation
Conversation
SparseLinOp now provides the optional materialize_cols hook (HasMaterializeCols) for CSR, CSC and COO, with the overflow-safe range check and the ldb >= n_rows check of the other providers. A zero count, or an operator with no rows, returns; nothing is allocated. Rows 0..n_rows-1 of the requested columns are zeroed and every stored entry in range is added, so duplicates are summed. CSC reads the requested columns directly; CSR scans every row in full (its column indices need not be sorted) and COO every entry. As in the sparse-dense products, a CSR or CSC operator must have its matrix's dimensions, a COO operator reads the leading n_rows x n_cols window of a larger matrix, and a one-based matrix is rejected (the sketch with a sparse S needs equal dimensions; see #274). Loops that touch at least kMaterializeColsParallelMin = 2^15 entries run with OpenMP: the zero fill over columns, CSC's fill and scatter over columns (counting the entries of the requested columns), CSR's scatter over rows (counting all its entries); COO's scatter stays serial. Each output entry is written by one thread, duplicates in storage order, so the result does not depend on the thread count. CompositeOperator's blocked sketch (block_size > 0, ColMajor, Side::Right, NoTrans) copies the right operand's columns through materialize_cols when the operand provides it (DenseLinOp and SparseLinOp are the tested cases), instead of applying it to identity columns, and then allocates no identity block. The composite's other requirements on its operands, their Side-taking and sketch products, still apply. CompositeOperator itself gets no materialize_cols: its columns need a product. The sketch operator's temporary buffers are now std::unique_ptr, so a throwing operand no longer leaks them. materialize's SparseLinOp overload (zero fill, then sparse_to_dense) is removed; a SparseLinOp takes the overload for operators with the hook. COO output is unchanged (coo_to_dense also adds in storage order onto zero). CSR and CSC output changes where sparse_to_dense disagreed with the operator's product: it kept the last of duplicate entries (now summed) and a stored -0.0 as -0.0 (now +0.0). The m x n temporary it allocated when ldb > m is gone. Numerics where a SparseLinOp's columns are now copied instead of multiplied by identity columns (the unpreconditioned first Gram of CholQR, CholQR2 and sCholQR3; the first product of sCholQR3_linops_basic; the test-mode Q of the CholQR family and of CQRRTO_linops; the sketch, first Gram, test-mode Q and materialize of [A; mu I]; the benchmark's KW reference; the composite sketch): for finite data with no duplicate entries each entry of such a product is one stored value plus exact zeros, so the copied sparse block is bit-identical to it, and the rest of each computation is unchanged. Both read a stored -0.0 and an empty entry as +0.0 (checked on MKL and on RandBLAS's kernels). The copy sums duplicates in storage order and the product in the backend's order, so two duplicates give the same sum and three or more can round differently. MKL's CSC product (mkl_sparse_d_mm, transposed, on the CSR of A^T) lost one of two duplicates stored next to each other in a column in probes here, where the copy, RandBLAS's CSC kernel and MKL's CSR and COO products sum them. An Inf or NaN stays in its own entry in the copy; the product turns the rest of its row in the block into NaN (Inf * 0). Subnormals are exact unless the floating-point environment flushes them (FTZ/DAZ), which affects the copy's additions and the product alike. In [A; mu I] with mu < 0, the mu I block's zeros were -0.0 from the product (mu * 0) and are +0.0 in the copy. A DenseLinOp right operand of the composite sketch has the numerics of DenseLinOp's hook. In the Q-less scaling study (CQRRTO_linop_basic) the CholQR and sCholQR3 rows copy the CSR operator's columns in their first Gram, so CholQR's fwd and sCholQR3's fwd1 slots time the copy, and CholQR's model (cholqr_linops_analytical_kb, through HasMaterializeCols) and peak drop the n x b identity block. Its check of Q and the CQRRT_expl row materialize through the hook: values are unchanged on its generated matrices and on Matrix Market files without repeated coordinates, and a file with repeated coordinates now gets their sum, as the operator's product does. CQRRTO_linops on a bare SparseLinOp changes only in its test-mode Q, and Blendenpik_linops on one not at all. No other benchmark output changes. test_helpers' ProductsOnly moves to RandLAPACK/testing/rl_test_utils.hh as RandLAPACK::testing::ProductsOnly, which forwards every product the operator accepts (requires std::invocable) and hides materialize_cols; the extras' Toeplitz tests use it instead of their own. The docstring of sparse_to_dense_summing_duplicates now says only COO sums duplicates. rl_materialize.hh drops includes it no longer needs. Tests: materialize_cols in each format, in double and float, on a matrix with a triple and a double duplicate, unsorted CSR and CSC indices and an empty row and column, at ranges at the start, middle and end and with zero counts, with padding rows untouched, equal to the product with identity columns there for CSR and COO and on duplicate-free random data for all three; a COO window of a larger matrix; a 16384 x 10 copy sized from kMaterializeColsParallelMin so that every loop is threaded, on at least 4 OpenMP threads; Inf, NaN and -0.0; materialize summing duplicates; the argument and shape checks; the one-based guard, on a view marked one-based by hand; an operator with no rows and a matrix with no stored entries. The composite sketch with a DenseLinOp and a SparseLinOp right operand counts that the copy path reads n columns and never applies the operand, and the product path applies it once per block. The sketch of [SparseLinOp; mu I], the CholQR family on a CSR SparseLinOp and on [SparseLinOp; mu I], CholQR_linops on a COO window and CQRRTO_linops on the stack match the same operators behind ProductsOnly. static_asserts check that ProductsOnly accepts exactly its operator's calls, and an explicit instantiation that TransposedLinOp still detects a missing Side-taking product through it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the CQRRTO split of #129 (#237-#248, #255, #262).
Summary
Gives
SparseLinOpthematerialize_colsmember (#241'sHasMaterializeCols), which writes chosen columns of an operator into a buffer without a product. The code that reads an operator's columns when it can (the CholQR-family first Gram, the test-mode Q, the[A; mu I]sketch,materialize) now copies a sparse matrix's stored entries instead of multiplying it by identity columns.CompositeOperator's blocked sketch also copies its right operand's columns when that operand has the member.Changes
linops/rl_sparse_linop.hh:materialize_colsfor CSR, CSC and COO. It zeroes the requested columns and adds every stored entry in range (duplicates summed in storage order, unsorted indices fine), allocates nothing, and threads with OpenMP abovekMaterializeColsParallelMinentries with results independent of the thread count. It checks range,ldband shape like the other providers, accepts a COO window of a larger matrix (as the COO products do), and rejects a one-based matrix, as the products do (the views fix is SparseLinOp: keep the matrix's index base; check the sparse sketch #274).linops/rl_composite_linop.hh: the blocked sketch fills each block fromright_op.materialize_colswhen available and then needs no identity block; its temporaries are nowstd::unique_ptr. The composite itself gets nomaterialize_cols, since its columns need a product.linops/rl_materialize.hh: the SparseLinOp overload ofmaterializeis removed; the overload for operators with the member makes the copies. For CSR and CSC, duplicate entries are now summed (the old overload kept the last) and a stored -0.0 reads as +0.0, both as the operator's own product does.[A; mu I]with mu < 0, the identity block's zeros are now +0.0. Probes here found that MKL's CSC product loses one of two adjacent duplicates in a column; the copy sums them.RandLAPACK::testing::ProductsOnly(forwards an operator's products and hidesmaterialize_cols) moves intotesting/rl_test_utils.hh, replacing a copy in the extras tests.TestSparseMaterializeCols(every format, duplicates, unsorted indices, windows, a threaded case, special values, argument checks), a composite sketch test that counts copies and products, a[SparseLinOp; mu I]sketch test, and the four CholQR-family drivers and CQRRTO on sparse operators against the product path. RandLAPACK_tests 742/742, extras 176/176, benchmark tree builds.