Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions lib/ecto/repo/schema.ex
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,11 @@ defmodule Ecto.Repo.Schema do
# On insert, we always merge the whole struct into the
# changeset as changes, except the primary key if it is nil.
changeset = put_repo_and_action(changeset, :insert, repo, tuplet)
changeset = Relation.surface_changes(changeset, struct, keep_fields ++ drop_fields ++ assocs)
changeset = Relation.surface_changes(changeset, struct, keep_fields ++ assocs)
# On insert, we need to nilify non-writable fields in
# the underlying data so that the returned struct reflects
# that the write was not actually performed.
changeset = update_in(changeset.data, &nilify_unsurfaced_non_writable_data!(&1, drop_fields, schema))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write this as:

Suggested change
changeset = update_in(changeset.data, &nilify_unsurfaced_non_writable_data!(&1, drop_fields, schema))
changeset = nilify_unsurfaced_non_writable_data!(changeset, drop_fields, schema)

And make it a no-op if drop_fields is an empty list, so we don't do work in the most common case (drop_fields is empty).

@green-david green-david Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I make the same changes to this?

changeset = update_in(changeset.changes, &drop_non_writable_changes!(&1, drop_fields, schema, :insert))

to

changeset = drop_non_writable_changes!(changeset, drop_fields, schema, :insert)

for consistency?

changeset = update_in(changeset.changes, &drop_non_writable_changes!(&1, drop_fields, schema, :insert))
Comment thread
greg-rychlewski marked this conversation as resolved.
Outdated

wrap_in_transaction(adapter, adapter_meta, opts, changeset, assocs, embeds, prepare, fn ->
Expand Down Expand Up @@ -522,6 +526,21 @@ defmodule Ecto.Repo.Schema do
{:error, put_repo_and_action(changeset, :insert, repo, tuplet)}
end

defp nilify_unsurfaced_non_writable_data!(data, non_writable_fields, schema) do
updates = Enum.reduce(non_writable_fields, [], fn field, updates ->
case data do
%{^field => value} when value != nil ->
handle_writable_violation(field, schema, :insert)

[{field, nil} | updates]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this how the previous code would behave? If we didn't surface it, I thought we would just leave the field value as is, no?

This is important because if a field has a default value different from nil, we will start setting it back to nil now.

@green-david green-david Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behavior is intentionally different as it appeared to be a bug:

#4524 proposed that the most obvious return value for a field whose value was not updated due to :writable would be the value actually committed to the database, and #4525 made the changes to do so, but only for actual changes, not surfaced changes.

This led to an odd discrepancy in the previous behavior:

# Returned struct does not reflect database state when inserted via a surfaced change
%{never: 10} = 
%SchemaWritable{never: 10}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert!()
# Returned struct does reflect database state when inserted via an actual change
%{never: nil} = 
%SchemaWritable{}
|> Ecto.Changeset.change(%{never: 10})
|> TestRepo.insert!()

This change aligns the behavior amongst the two so at least we are consistent, as the former certainly appears at first glance to be a bug.

If the field has a default value different from nil, and we go to insert the struct, and the field is writable :never, we wouldn't have inserted the default value, so should we not return nil to reflect that? (That's the idea anyways)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the field has a default value different from nil, and we go to insert the struct, and the field is writable :never, we wouldn't have inserted the default value, so should we not return nil to reflect that? (That's the idea anyways)

The problem with this assumption is that it assumes nil is a reasonable value for the field, which may not be the case. If you declare a field with a default of 0, now we are inserting nil where no nil was ever expected. :(

At the same time, if said field is marked as non-writable, then it means it will always warn, regardless of what the value is, forcing users to use nil when an actual value such as 0 may be better.

So I think the best solution overall may be to not warn nor change the value at all in those cases. :(

@green-david green-david Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with this assumption is that it assumes nil is a reasonable value for the field, which may not be the case. If you declare a field with a default of 0, now we are inserting nil where no nil was ever expected. :(

Was this not already the case due to the fact that we filtered out all of the changes before the write silently? I can double check. I think it would get set to the default value in the database, which wouldn't get returned previously unless :returning is specified so im not sure the actual write behavior actually changed? (I could be misunderstanding you).

At the same time, if said field is marked as non-writable, then it means it will always warn, regardless of what the value is, forcing users to use nil when an actual value such as 0 may be better.

Yeah, there is a definite not-intuitive conflict between default and writable. Greg and I discussed it a bit in the original PR here.

It wouldn't have to always warn, no? If they configured on_writable_violation: :nothing (which is the default still), it would continue to behave the same (I think?)

So I think the best solution overall may be to not warn nor change the value at all in those cases. :(

This would certainly be unfortunate. If we do end up having to go this route where surfaced changes can't be detected as writable violations, I guess the best we can do is document it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this not already the case due to the fact that we filtered out all of the changes before the right silently? I can double check.

We don't use database values unless you opt-in via returning. In that case, it makes sense whatever is returned from the database is always ignored.

It wouldn't have to always warn, no? If they configured on_writable_violation: :nothing (which is the default still), it would continue to behave the same (I think?)

You are right, given you can always opt-out at by saying nothing, then that concern is fine (which is the same conclusion you arrived, sorry for being a slow poke). I'd still say though writable changes the behaviour of default, which we should probably avoid. The good news is that it would be just removing code!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josevalim Would you mind humoring me with a small summary of what you are suggesting as the path forward? I am happy to make the changes; I just want to be 100% sure I understand what you are proposing before doing so. 🙏

%{} ->
updates
end
end)

struct!(data, updates)

@josevalim josevalim Jun 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am propsing something like this:

Suggested change
updates = Enum.reduce(non_writable_fields, [], fn field, updates ->
case data do
%{^field => value} when value != nil ->
handle_writable_violation(field, schema, :insert)
[{field, nil} | updates]
%{} ->
updates
end
end)
struct!(data, updates)
if non_writeble_fields != [] and on_writable_violation_for_field(field) != :nothing do
Enum.each(non_writable_fields, fn field ->
case data do
%{^field => value} when value != nil ->
handle_writable_violation(field, schema, :insert)
_ ->
updates
end
end)
end

But I am not sure, so I appreciate all of the push back!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed an enhancement to the branch which meaqns on_writable_violation_for_field(field) is the same as module.__schema__(:on_writable_violation, field) :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josevalim Thank you! Just double-checking that by "the branch" you did intend to push to master, which is where that got committed, and not this branch 😉

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct!!!

PS: I just want to say that interacting with you on all of these threads has been fantastic, thank you! ❤️

end

@doc """
Implementation for `Ecto.Repo.update/2`.
"""
Expand Down Expand Up @@ -630,7 +649,8 @@ defmodule Ecto.Repo.Schema do
case changes do
%{^field => _change} ->
handle_writable_violation(field, schema, action)
changes

Map.delete(changes, field)
%{} ->
changes
end
Expand Down
3 changes: 1 addition & 2 deletions lib/ecto/schema.ex
Original file line number Diff line number Diff line change
Expand Up @@ -706,8 +706,7 @@ defmodule Ecto.Schema do
attempts to modify a field that should not be modified according to it's `:writable` value.
Must be one of `:nothing`, `:warn`, or `:raise`. If set to `:nothing`, the modification is
silently ignored. If set to `:warn`, the modification is ignored and a warning is logged. If set
to `:raise`, an exception is raised and the operation is aborted. If `:writable` is set to `:always`,
`:on_writable_violation` must be set to `:nothing`. Defaults to `:nothing`.
to `:raise`, an exception is raised and the operation is aborted. Defaults to `:nothing`.

"""
defmacro field(name, type \\ :string, opts \\ []) do
Expand Down
59 changes: 33 additions & 26 deletions test/ecto/repo_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -2440,18 +2440,20 @@ defmodule Ecto.RepoTest do
end

test "update with on_writable_violation: :nothing saves changes for writable: :always and ignores changes for writable: :insert/:never" do
%MySchemaWritable{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.update()
%{always: 10, never: nil, insert: nil} =
%MySchemaWritable{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.update!()

assert_received {:update, %{changes: [always: 10]}}
end

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
test "update with on_writable_violation: :nothing saves changes for writable: :always and ignores changes for writable: :insert/:never" do
%{never: nil} =
%MySchemaWritable{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.update!()
assert_received {:update, %{changes: [always: 10]}}
end

I think this is enough and then you can also make this minimal change to the existing insert test.


test "update with on_writable_violation: :warn saves changes for writable: :always, ignores changes for writable: :insert/:never, and logs a warning" do
log = capture_log(fn ->
%MySchemaWritableWarn{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.update()
%{always: 10, never: nil, insert: nil} =
%MySchemaWritableWarn{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.update!()

assert_received {:update, %{changes: [always: 10]}}
end)
Expand All @@ -2464,18 +2466,18 @@ defmodule Ecto.RepoTest do
assert_raise ArgumentError, "attempted to write to non-writable field :never during update", fn ->
%MySchemaWritableRaise{id: 1}
|> Ecto.Changeset.change(%{never: 10})
|> TestRepo.update()
|> TestRepo.update!()
end

assert_raise ArgumentError, "attempted to write to non-writable field :insert during update", fn ->
%MySchemaWritableRaise{id: 2}
|> Ecto.Changeset.change(%{insert: 11})
|> TestRepo.update()
|> TestRepo.update!()
end

%MySchemaWritableRaise{id: 3}
|> Ecto.Changeset.change(%{always: 12})
|> TestRepo.update()
|> TestRepo.update!()

assert_received {:update, %{changes: [always: 12]}}
end
Expand Down Expand Up @@ -2514,28 +2516,31 @@ defmodule Ecto.RepoTest do
end

test "insert with surfaced changes on_writable_violation: :nothing saves changes for writable: :always/:insert and ignores changes for writable: :never" do
%MySchemaWritable{id: 1, always: 10, never: 11, insert: 12}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert()
%{always: 10, never: nil, insert: 12} =
%MySchemaWritable{id: 1, always: 10, never: 11, insert: 12}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12]
end

test "insert with on_writable_violation: :nothing saves changes for writable: :always/:insert and ignores changes for writable: :never" do
%MySchemaWritable{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.insert()
%{always: 10, never: nil, insert: 12} =
%MySchemaWritable{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12]
end

test "insert with with surfaced changes and on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do
test "insert with surfaced changes and on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do
log = capture_log(fn ->
%MySchemaWritableWarn{id: 1, always: 10, never: 11, insert: 12}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert()
%{always: 10, never: nil, insert: 12} =
%MySchemaWritableWarn{id: 1, always: 10, never: 11, insert: 12}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12]
Expand All @@ -2546,9 +2551,10 @@ defmodule Ecto.RepoTest do

test "insert with on_writable_violation: :warn saves changes for writable: :always/:insert, ignores changes for writable: :never, and logs a warning" do
log = capture_log(fn ->
%MySchemaWritableWarn{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.insert()
%{always: 10, never: nil, insert: 12} =
%MySchemaWritableWarn{id: 1}
|> Ecto.Changeset.change(%{always: 10, never: 11, insert: 12})
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 10, id: 1, insert: 12]
Expand All @@ -2561,12 +2567,13 @@ defmodule Ecto.RepoTest do
assert_raise ArgumentError, "attempted to write to non-writable field :never during insert", fn ->
%MySchemaWritableRaise{id: 1, never: 10}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert()
|> TestRepo.insert!()
end


%MySchemaWritableRaise{id: 2, insert: 11, always: 12}
|> Ecto.Changeset.change(%{})
|> TestRepo.insert()
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 12, id: 2, insert: 11]
Expand All @@ -2576,12 +2583,12 @@ defmodule Ecto.RepoTest do
assert_raise ArgumentError, "attempted to write to non-writable field :never during insert", fn ->
%MySchemaWritableRaise{id: 1}
|> Ecto.Changeset.change(%{never: 10})
|> TestRepo.insert()
|> TestRepo.insert!()
end

%MySchemaWritableRaise{id: 2}
|> Ecto.Changeset.change(%{insert: 11, always: 12})
|> TestRepo.insert()
|> TestRepo.insert!()

assert_received {:insert, %{fields: inserted_fields}}
assert Enum.sort(inserted_fields) == [always: 12, id: 2, insert: 11]
Expand Down
Loading