DZone Spotlight🔒 SSL Verified

"Onboarding a new table into row-level security should be four lines of metadata. Not two new objects, a code review, and a platform-team ticket. This post describes a tag-driven attribute-based access control (ABAC) pattern built on Databricks Unity Catalog primitives that achieves the objective of one UDF per filter shape, one policy per shape, and a single control table that drives all per-group authorization logic. I work as a solutions architect with large enterprises running hundreds of tables across multiple regions, product lines, and source systems, where row-level security follows a pattern: Group A sees records from System X. Group B sees the regions China and India. Group C sees plant key 333. Group D combines two source systems. Group E sees everything except specific values. The domain (finance, healthcare, etc.) doesn't matter. The pattern remains the same. A traditional implementation looks like this: One row-filter UDF per tableOne row-filter policy per (table and group) combinationA SQL query layer that joins every table to an identity mapping table Every new table required writing two new objects (UDF + policy) and updating every existing group definition. Every new group required touching every UDF. Onboarding a new product line meant rebuilding the whole machine. Hundreds of tables × dozens of groups = a maintenance nightmare. Instead, what they needed was a pattern where: A new table joins the RLS scheme with only metadata changes — no new UDFs, no new policiesA new business group is just a data write — no DDL, no code reviewMisconfigured rules fail closed, not open This post describes the four-layer pattern we landed on. It's built entirely on Unity Catalog ABAC primitives (governed tags, row-filter policies, attribute-based binding) and a single control table that drives per-group filter logic. The Four Layers Each layer does exactly one thing and one thing only: Layer 1: Tags are declarative metadata. A table-level tag declares which shape a table is. A column-level tag tells the row filter which physical column corresponds to which logical attribute. All tags do is describe the data. They facilitate the action for the next layers.Layer 2: UDF is the decision logic. Given a row's attribute values, it returns a boolean answer of TRUE or FALSE for current_user(). It doesn't know anything about which table it's filtering; it just answers "is this row visible?"Layer 3: Policy is the binding. It says, "for tables tagged with shape X, call UDF Y with these columns." It uses tag-matching expressions so it auto-attaches to new tables as they're tagged. This is what enables us to avoid per-table DDL. Layer 4: Row-Level Security is what the customer experiences. Their SQL doesn't change; filtered rows just come back. Layer 1 + 2: Tags Describe, UDFs Decide Two governed tags do all the work. A rls_tag on the table says "this is a table_1_filter shape." An rls_attr tag on each column says "this column is the src_sys_cd attribute" or "this column is the region attribute." The column tag is the load-bearing piece — it lets you have a column literally named ws_region_cd and still have the policy treat it as the logical region attribute. Physical naming is decoupled from policy semantics. One UDF per table shape. A "shape" is a set of filterable columns. In this customer's setup, there are two shapes: Table 1 shape: (src_sys_cd, plant_key, region) — three attributes. Maybe it's best to call this shape a combination of the keys. For example, all the tables that have src_sys_cd + plant_key+region fall under this shape.Table 2 shape: (src_sys_cd, order_key) — two attributes Each shape has its own UDF (rf_table_1, rf_table_2). The UDF signature takes one parameter per filterable column. The body joins a user_group_control_table (one row per group rule) to a user_group_membership mapping (or, in production, is_account_group_member()) and applies BOOL_OR across the rules — a union grant. The key property of this UDF design: it doesn't know which table is calling it. It just answers, given attribute values, "does current_user() get this row?" That's what lets the same UDF serve many tables of the same shape. Layer 3: The Policy that ties it all together The policy is the only object in the system that knows about both tags and UDFs. Its four clauses each answer a separate question: clausequestion it answers `ON SCHEMA …` Where does this policy live? (Schema-scoped — broad reach.) `WHEN has_tag_value('rls_tag', '…')` Which tables should it attach to? Anything tagged with the right shape. `MATCH COLUMNS … has_tag_value('rls_attr', '…')` Inside each table, which physical column maps to which logical attribute? `ROW FILTER … USING COLUMNS (…)` Which UDF to call, and in what argument order? The auto-attachment behavior is what makes the pattern scale. The policy doesn't enumerate tables — it matches them by tag. Tag a new table tomorrow, and the policy applies to it on the next query. Zero policy edits. Query Time: What Actually Happens When a user runs SELECT * FROM table_1, here's what Unity Catalog does behind the scenes: The planner sees the table and looks up policies attached to its schema. It finds rls_policy_t1.The policy's `WHEN` clause checks the table tag. Does table_1 have rls_tag=table_1_filter? Yes → the policy attaches. (If no, the policy is skipped for this table.)The policy's `MATCH COLUMNS` resolves attributes. For each logical attribute name, it scans column tags to find the physical column with that role: src_sys_cd → physical column src_sys_cd; plant_key → physical column plant_key; region → physical column region.The query is rewritten to append WHERE rf_table_1(src_sys_cd, plant_key, region) = TRUE. The customer's original SQL is unchanged.The UDF runs per row. It joins the control table to membership for current_user(), evaluates each rule, and BOOL_ORs the results — TRUE if any rule grants the row, FALSE otherwise.The engine emits only the `TRUE` rows. The customer sees only their authorized subset. They never see the UDF call or the policy mechanics. The whole thing is transparent to the application — same SQL, filtered result. The Scale Payoff The reason to build the pattern this way only becomes obvious when you onboard the second, third, and hundredth table. Adding a New Table to an Existing Shape A new dimension table arrives that fits the Table 1 shape. The work to bring it under RLS: Stepeffort shape Create the table (customer's normal DDL) — Tag the table with the shape 1 line of DDL Tag the columns with their logical roles 3 lines of DDL Update the UDF? None Update the policy? None Update the control table? **None** (existing groups apply automatically through their existing rules) Four ALTER lines. That's the whole onboarding cost. Adding a New Business Group A new business group needs access to a specific slice of the data: stepeffort `INSERT` one row into `user_group_control_table` 1 INSERT Add users to the AD group (outside Databricks) — Update the UDF? None Update the policy? None Update any table tags? None One INSERT. Adding a new group is a data write, not DDL — which means operations teams can self-serve through their normal change-management process, without code review or platform-team involvement. GitHub repo: https://github.com/vbablue/databricks-abac-rls-demo/tree/main"

"My converter has a test suite. It was green. Every example I could think of went in, the right pytest file came out, and I shipped it to PyPI as 1.0. Weeks later it sits there marked Production/Stable. Then I pointed a fuzzer at it, and it stopped being so quiet. The Blind Spot in Example Tests Unit tests check the inputs you thought of. That is their whole nature: you sit down, you imagine how the code gets used, you write an example for each case. The cases you imagine tend to be the cases the code already handles, because you wrote the code and the test in the same afternoon, with the same blind spots. A converter makes that worse. postman2pytest takes a Postman collection, a JSON file, and turns it into a runnable pytest suite. The input is not something I control. It is a file a stranger exports from a tool, edits by hand, truncates over a flaky download, or generates from a script that had a bug. The real test suite for a parser is the set of files you did not write. So I stopped writing examples and let a machine write them for me. Letting Hypothesis Do the Imagining Hypothesis (hypothesis.readthedocs.io) is property-based testing for Python. Instead of one example, you describe the shape of the input and assert a property that must hold for all of them. It then generates hundreds of cases, including the mean ones you would never type out. For a parser, the property is simple to state. Feed it any well-formed JSON, whatever its shape. It should do exactly one of two things: return a list of parsed tests, or raise a clear error that tells the user what is wrong. It must never fall over with a raw stack trace. JSON import json from hypothesis import given, strategies as st json_values = st.recursive( st.none() | st.booleans() | st.integers() | st.text(), lambda children: st.lists(children) | st.dictionaries(st.text(), children), ) @given(value=json_values) def test_parser_is_graceful_on_any_json(value, tmp_path): path = tmp_path / "in.json" path.write_text(json.dumps(value)) try: result = parse_collection(path) except ValueError: return # the one allowed failure: a clear, typed error assert isinstance(result, list) Then I ran it. What It Found The parser handled every real Postman collection the same as before. The fuzzer never touched valid input. It broke my handling of garbage, in five distinct shapes. A collection whose top level was a list instead of an object. A collection whose info block was a string. An item that was a bare number where the code expected a dict. A folder nested deep enough that the recursion bottomed out. And a payload so deeply nested that json.loads itself blew the stack before my code ran at all. In each case the user would have seen an AttributeError, a TypeError, or a RecursionError, with an internal stack trace and no idea what to fix. None of those is a useful message. The tool knew the input was wrong. It just had no manners about saying so. What sold me on the method was the shrinking. When Hypothesis finds a failure, it does not hand you the random 4KB blob that happened to trip the code. It shrinks the case to the smallest input that still fails, so the report is a two-line collection, not a wall of generated noise. That is the difference between a bug you can read and a bug you file under "investigate later". Each of the five failures arrived as a minimal reproducer I could paste straight into a regression test, which is how plugging them stayed a one-afternoon job rather than a week of bisecting. The Fix Is a Contract, Not a Patch Plugging the five holes was the boring part. The contract underneath them was the real work: valid collection in, pytest suite out; anything else, one clear ValueError that names the problem. Degrade, do not crash. That turned into a few isinstance checks at the boundaries, a depth limit on folder recursion, and wrapping the top-level json.loads so a stack-busting document becomes a readable "this file is nested too deeply to parse" instead of a traceback. The same review on my other parser, secure-log2test, which reads Kibana log exports, turned up the same family of gaps in the same afternoon. Same shape of bug, same shape of fix. What I Actually Learned "Production/Stable" is a statement about the API, not about the input. It never promised to survive a corrupt file, and I had let people assume it did. And finding bugs with a fuzzer is the fuzzer doing its job. Point Hypothesis at a parser and it finds nothing? You probably did not let it generate anything nasty enough. The bugs are the evidence the method works. If your tool reads a file someone else made, your unit tests are a comfort blanket. Spend twenty lines on a property test before you trust them. The input you did not write is the one that finds you in production. postman2pytest and secure-log2test are MIT-licensed on PyPI. The fuzz tests are in each repo if you want a template to copy. This article was originally published on dev.to."