-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Do not reflect changes ignored due to :writable in returned struct #4736
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
401efa8
bfe9ab7
58413a6
73fac16
51183d8
f4bb632
241b3ee
f397115
5f72973
7337183
c74527b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -452,7 +452,11 @@ defmodule Ecto.Repo.Schema do | |||||||||||||||||||||||||||||||||||||||||||||
| # On insert, we always merge the whole struct into the | ||||||||||||||||||||||||||||||||||||||||||||||
| # changeset as changes, except the primary key if it is nil. | ||||||||||||||||||||||||||||||||||||||||||||||
| changeset = put_repo_and_action(changeset, :insert, repo, tuplet) | ||||||||||||||||||||||||||||||||||||||||||||||
| changeset = Relation.surface_changes(changeset, struct, keep_fields ++ drop_fields ++ assocs) | ||||||||||||||||||||||||||||||||||||||||||||||
| changeset = Relation.surface_changes(changeset, struct, keep_fields ++ assocs) | ||||||||||||||||||||||||||||||||||||||||||||||
| # On insert, we need to nilify non-writable fields in | ||||||||||||||||||||||||||||||||||||||||||||||
| # the underlying data so that the returned struct reflects | ||||||||||||||||||||||||||||||||||||||||||||||
| # that the write was not actually performed. | ||||||||||||||||||||||||||||||||||||||||||||||
| changeset = update_in(changeset.data, &nilify_unsurfaced_non_writable_data!(&1, drop_fields, schema)) | ||||||||||||||||||||||||||||||||||||||||||||||
| changeset = update_in(changeset.changes, &drop_non_writable_changes!(&1, drop_fields, schema, :insert)) | ||||||||||||||||||||||||||||||||||||||||||||||
|
greg-rychlewski marked this conversation as resolved.
Outdated
|
||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| wrap_in_transaction(adapter, adapter_meta, opts, changeset, assocs, embeds, prepare, fn -> | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
@@ -522,6 +526,21 @@ defmodule Ecto.Repo.Schema do | |||||||||||||||||||||||||||||||||||||||||||||
| {:error, put_repo_and_action(changeset, :insert, repo, tuplet)} | ||||||||||||||||||||||||||||||||||||||||||||||
| end | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| defp nilify_unsurfaced_non_writable_data!(data, non_writable_fields, schema) do | ||||||||||||||||||||||||||||||||||||||||||||||
| updates = Enum.reduce(non_writable_fields, [], fn field, updates -> | ||||||||||||||||||||||||||||||||||||||||||||||
| case data do | ||||||||||||||||||||||||||||||||||||||||||||||
| %{^field => value} when value != nil -> | ||||||||||||||||||||||||||||||||||||||||||||||
| handle_writable_violation(field, schema, :insert) | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| [{field, nil} | updates] | ||||||||||||||||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this how the previous code would behave? If we didn't surface it, I thought we would just leave the field value as is, no? This is important because if a field has a default value different from nil, we will start setting it back to nil now.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The behavior is intentionally different as it appeared to be a bug: #4524 proposed that the most obvious return value for a field whose value was not updated due to This led to an odd discrepancy in the previous behavior: # Returned struct does not reflect database state when inserted via a surfaced change
%{never: 10} =
%SchemaWritable{never: 10}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert!()# Returned struct does reflect database state when inserted via an actual change
%{never: nil} =
%SchemaWritable{}
|> Ecto.Changeset.change(%{never: 10})
|> TestRepo.insert!()This change aligns the behavior amongst the two so at least we are consistent, as the former certainly appears at first glance to be a bug. If the field has a default value different from nil, and we go to insert the struct, and the field is writable
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The problem with this assumption is that it assumes nil is a reasonable value for the field, which may not be the case. If you declare a field with a default of At the same time, if said field is marked as non-writable, then it means it will always warn, regardless of what the value is, forcing users to use So I think the best solution overall may be to not warn nor change the value at all in those cases. :(
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Was this not already the case due to the fact that we filtered out all of the changes before the write silently? I can double check. I think it would get set to the default value in the database, which wouldn't get returned previously unless
Yeah, there is a definite not-intuitive conflict between It wouldn't have to always warn, no? If they configured
This would certainly be unfortunate. If we do end up having to go this route where surfaced changes can't be detected as writable violations, I guess the best we can do is document it.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
We don't use database values unless you opt-in via returning. In that case, it makes sense whatever is returned from the database is always ignored.
You are right, given you can always opt-out at by saying nothing, then that concern is fine (which is the same conclusion you arrived, sorry for being a slow poke). I'd still say though writable changes the behaviour of
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @josevalim Would you mind humoring me with a small summary of what you are suggesting as the path forward? I am happy to make the changes; I just want to be 100% sure I understand what you are proposing before doing so. 🙏 |
||||||||||||||||||||||||||||||||||||||||||||||
| %{} -> | ||||||||||||||||||||||||||||||||||||||||||||||
| updates | ||||||||||||||||||||||||||||||||||||||||||||||
| end | ||||||||||||||||||||||||||||||||||||||||||||||
| end) | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| struct!(data, updates) | ||||||||||||||||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am propsing something like this:
Suggested change
But I am not sure, so I appreciate all of the push back!
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I pushed an enhancement to the branch which meaqns
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @josevalim Thank you! Just double-checking that by "the branch" you did intend to push to
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct!!! PS: I just want to say that interacting with you on all of these threads has been fantastic, thank you! ❤️ |
||||||||||||||||||||||||||||||||||||||||||||||
| end | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| @doc """ | ||||||||||||||||||||||||||||||||||||||||||||||
| Implementation for `Ecto.Repo.update/2`. | ||||||||||||||||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
@@ -630,7 +649,8 @@ defmodule Ecto.Repo.Schema do | |||||||||||||||||||||||||||||||||||||||||||||
| case changes do | ||||||||||||||||||||||||||||||||||||||||||||||
| %{^field => _change} -> | ||||||||||||||||||||||||||||||||||||||||||||||
| handle_writable_violation(field, schema, action) | ||||||||||||||||||||||||||||||||||||||||||||||
| changes | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| Map.delete(changes, field) | ||||||||||||||||||||||||||||||||||||||||||||||
| %{} -> | ||||||||||||||||||||||||||||||||||||||||||||||
| changes | ||||||||||||||||||||||||||||||||||||||||||||||
| end | ||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -2440,18 +2440,20 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| test "update with on_writable_violation: :nothing saves changes for writable: :always and ignores changes for writable: :insert/:never" do | ||||||||||||||||||||
| %MySchemaWritable{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.update() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: nil} = | ||||||||||||||||||||
| %MySchemaWritable{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.update!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:update, %{changes: [always: 10]}} | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
I think this is enough and then you can also make this minimal change to the existing insert test. |
||||||||||||||||||||
|
|
||||||||||||||||||||
| test "update with on_writable_violation: :warn saves changes for writable: :always, ignores changes for writable: :insert/:never, and logs a warning" do | ||||||||||||||||||||
| log = capture_log(fn -> | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.update() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: nil} = | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.update!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:update, %{changes: [always: 10]}} | ||||||||||||||||||||
| end) | ||||||||||||||||||||
|
|
@@ -2464,18 +2466,18 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
| assert_raise ArgumentError, "attempted to write to non-writable field :never during update", fn -> | ||||||||||||||||||||
| %MySchemaWritableRaise{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{never: 10}) | ||||||||||||||||||||
| |> TestRepo.update() | ||||||||||||||||||||
| |> TestRepo.update!() | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_raise ArgumentError, "attempted to write to non-writable field :insert during update", fn -> | ||||||||||||||||||||
| %MySchemaWritableRaise{id: 2} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{insert: 11}) | ||||||||||||||||||||
| |> TestRepo.update() | ||||||||||||||||||||
| |> TestRepo.update!() | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| %MySchemaWritableRaise{id: 3} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 12}) | ||||||||||||||||||||
| |> TestRepo.update() | ||||||||||||||||||||
| |> TestRepo.update!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:update, %{changes: [always: 12]}} | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
@@ -2514,28 +2516,31 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| test "insert with surfaced changes on_writable_violation: :nothing saves changes for writable: :always/:insert and ignores changes for writable: :never" do | ||||||||||||||||||||
| %MySchemaWritable{id: 1, always: 10, never: 11, insert: 12} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: 12} = | ||||||||||||||||||||
| %MySchemaWritable{id: 1, always: 10, never: 11, insert: 12} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12] | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| test "insert with on_writable_violation: :nothing saves changes for writable: :always/:insert and ignores changes for writable: :never" do | ||||||||||||||||||||
| %MySchemaWritable{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: 12} = | ||||||||||||||||||||
| %MySchemaWritable{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12] | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| test "insert with with surfaced changes and on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do | ||||||||||||||||||||
| test "insert with surfaced changes and on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do | ||||||||||||||||||||
| log = capture_log(fn -> | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1, always: 10, never: 11, insert: 12} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: 12} = | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1, always: 10, never: 11, insert: 12} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12] | ||||||||||||||||||||
|
|
@@ -2546,9 +2551,10 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
|
|
||||||||||||||||||||
| test "insert with on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do | ||||||||||||||||||||
| log = capture_log(fn -> | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| %{always: 10, never: nil, insert: 12} = | ||||||||||||||||||||
| %MySchemaWritableWarn{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12}) | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12] | ||||||||||||||||||||
|
|
@@ -2561,12 +2567,13 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
| assert_raise ArgumentError, "attempted to write to non-writable field :never during insert", fn -> | ||||||||||||||||||||
| %MySchemaWritableRaise{id: 1, never: 10} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| %MySchemaWritableRaise{id: 2, insert: 11, always: 12} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 12, id: 2, insert: 11] | ||||||||||||||||||||
|
|
@@ -2576,12 +2583,12 @@ defmodule Ecto.RepoTest do | |||||||||||||||||||
| assert_raise ArgumentError, "attempted to write to non-writable field :never during insert", fn -> | ||||||||||||||||||||
| %MySchemaWritableRaise{id: 1} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{never: 10}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
| end | ||||||||||||||||||||
|
|
||||||||||||||||||||
| %MySchemaWritableRaise{id: 2} | ||||||||||||||||||||
| |> Ecto.Changeset.change(%{insert: 11, always: 12}) | ||||||||||||||||||||
| |> TestRepo.insert() | ||||||||||||||||||||
| |> TestRepo.insert!() | ||||||||||||||||||||
|
|
||||||||||||||||||||
| assert_received {:insert, %{fields: inserted_fields}} | ||||||||||||||||||||
| assert Enum.sort(inserted_fields) == [always: 12, id: 2, insert: 11] | ||||||||||||||||||||
|
|
||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Write this as:
And make it a no-op if drop_fields is an empty list, so we don't do work in the most common case (drop_fields is empty).
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall I make the same changes to this?
to
for consistency?