DEV Community

Fetch returns an array, but javascript doesn't recognise fields.

Simone Aronica on February 06, 2020

Background I'm creating a website for my school using react and node.js. Now when I fetch all the articles the fetch actually sends all ...

Read full post

Daniel Brady • Feb 6 '20

Hi @itssimonedev, can you clarify what problem you are having?

I understand you are fetching some results from an API, and getting them back in an array of JS objects.

The title of your post says something about "JS doesn't recognize fields," and your post itself says something about not having all the data, but it's not clear what issue you are having.

Maybe sharing a snippet of code that:

shows the data you are trying to access
how you are accessing it
your expected vs. actual results

can help us help you 😄

Simone Aronica • Feb 6 '20 • Edited

Yes, you're right, I should have given a bit more of explanation.
I'm retrieving some data about articles in a database.

const ArticleSchema = new Schema({
  title: String,
  author: {
    type: Schema.Types.ObjectId,
    ref: "User"
  },
  body: String,
  tags: [String],
  created: {
    type: Date,
    default: Date.now
  },
  published: {
    type: Boolean,
    default: false
  }
});

In this exact case I need to render a list with all the articles inside the database (Only two at the moment) so from a restful route I fetch those articles:

async function _listAllArticles({offset = 0, limit = 10}) {
  const articles = await ArticleModel.find({ published: true })
    .select("-published")
    .sort({ created: "asc" })
    .skip(offset)
    .limit(limit);
  return articles;
}

...

app.get("/api/article/fetchlatest", (req, res) => {
  const offset = parseInt(req.query.offset) || 0;
  article._listAllArticles({offset: offset}).then(list => res.json(list));
});

and this is the response:

so I get all the data I need from fetching.

Now to the problem, this is the snippet of the fetch:

fetch(`/api/article/fetchlatest?offset=${offset}`)
      .then(res => res.json())
      .then(res => {
        console.log(res);
        setList(res);
        setFetched(true);
        setOffset(offset + 10);
      });

After that res.json() even though checking in the network tab I get two elements in the array, the resulting array from res.json() is:

so basically I retrieve n elements from the array, but that res.json() always returns an array of x elements where x is always less than the expected n value.

The weirdest thing is that you can see from the second picture that the minimized version of the console.log is (2)[{...}, {...}], but upon expanding you can only see one field in the array and the length is 1 instead of 2.

I hope this helps in explaining my problem. Basically, I fetch some data and when I res.json() that data, it simply won't return the whole array.

Daniel Brady • Feb 6 '20 • Edited

Thanks Simone, that's much clearer! Strange indeed 🤔

One theory I have is that the fetch is working properly, but one of the functions after your console.log(res) is actually modifying the contents of res, causing one of the results to be removed.

The basis of my theory is on that dev console weirdness: Chome lazily evaluates logged objects (I think that little blue info tag on the output says something about it, IIRC). So I think res has two objects when you log it out, but only one object by the time you expand the log details in the console 😬

I could definitely be wrong here, but it's easy to verify: just remove those set* statements after the log and see what's in the console. 🤞

Simone Aronica • Feb 6 '20

That's why I can't stop thinking about it... That setList is the only one in the code, and since list is a const I couldn't set it without using hooks anyways. There is literally nothing touching list after that setList... It's... Crazy

Daniel Brady • Feb 6 '20

So are you saying you tried removing the set* calls and it still didn't work?

Also, const doesn't impact your ability to change inherently mutable data structures like arrays: it just prevents variable assignments. So anything could be changing your list if you give them a handle to it. Simple example:

If you want to understand const a bit deeper, I recently wrote a post about it:

Using JS: const

Daniel Brady ・ Jan 20 ・ 10 min read

#javascript #thinkdeep #spec

Simone Aronica • Feb 6 '20 • Edited

I have no set calls to remove, because the only setList call that I have is in that section I've shown you. I was wondering on whether a list = something could have changed those values but since I used the hook const [list, setList] = useState() list can only be modified through setList, and its only instance is the one I've shown. Also the problem is far beyond that, because I'm not logging the list, but the result res.json() itself, so it's as if it had not the time to process everything (Impossible, since res.json() returns a promise...

chico1992 • Feb 6 '20 • Edited

can you try following code and tell us the what you get in the console

fetch(`/api/article/fetchlatest?offset=${offset}`)
      .then(res => res.json())
      .then(res => {
        console.log(res);
        setList([...res]);
        setFetched(true);
        setOffset(offset + 10);
      });

if your console.log spits out the right thing you are somewhere mutating your list

Edit: based on your screenshots it looks like you have a list.shift() somewhere

Daniel Brady • Feb 6 '20 • Edited

Thanks for illustrating this @chico1992 🙏 That's the point I was trying to articulate.

I just did a search and found that at some point, React 16.7(ish?) made a change which uses an object identity check in the implementation of useState; the result is that the setter will do nothing if it is called with the same object that is currently memoized.

This is relevant because it implies that the object given to the setter is stored by reference, not copied and stored by value, and thus can be mutated without interacting with the setter as long as you have a reference to it. 😓

Simone Aronica • Feb 7 '20

Ok so you're saying that since it's memorized as reference I could copy the array - not the reference - generated by the .json() into a new variable and then use setList to insert the reference of the new object into the state, instead of directly setting it?

chico1992 • Feb 7 '20

Yes you should always pass a new state to the set state and not mutate it i guess your useing a method somewhere that mutates the array stored in your state
Since hooks are based on functional principels you should be very careful with mutating your state
Const doesn’t prevent mutating an array or an object

Simone Aronica • Feb 7 '20

Okay, new tests led to this:
Now when I console.log([...res]) the array shows both of the fields, but then when I setList([...res]); console.log(list) it shows only the first value... I have no idea of the reason.
I've created a pastebin with the whole code: pastebin.com/AAkyJusJ
Also this is the image of what my console is now:
Don't mind those errors as they're not about the code itself, but they're generated by react-md-components.

chico1992 • Feb 7 '20

the error is the following line since splice muatest the array
splice doc
const elem = list.splice(i, i + 1)[0];

btw you should run your fetch logic in a useEffect since it's a side effect

Daniel Brady • Feb 7 '20

Ah! There's your problem on line 56:

const elem = list.splice(i, i + 1)[0];

The splice method is a mutator: it changes what's in your list state!

Here's the doc for details, but basically you need to use a different approach for extracting the element you want there.

I think slice should work:
developer.mozilla.org/en-US/docs/W...

chico1992 • Feb 7 '20 • Edited

or even
const elem = list[i]
if I'm not mistaken

Daniel Brady • Feb 7 '20 • Edited

Yeah definitely! 👏

Simone Aronica • Feb 8 '20

Oh God I though I was using .slice... All this trouble for a p.. I can't believe it.

Thanks to both of you, you were life saver, I would have probably rewritten the entire code to find this.

Jen • Feb 6 '20

when you expand the object at index 0, is the "missing" data still in there? I wonder if it's because you're responding with the stringified list. Maybe try console.log(JSON.parse(res)) and compare to console.log(res)